-- Logs begin at Sat 2022-10-15 08:06:22 UTC, end at Sat 2022-10-15 09:34:12 UTC. -- Oct 15 08:30:58.732252 dv-cinder-856971-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 15 08:30:58.743372 dv-cinder-856971-3-iscsi devstack@keystone.service[88051]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 15 08:30:58.743372 dv-cinder-856971-3-iscsi devstack@keystone.service[88051]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sat Oct 15 08:30:58 2022] *** Oct 15 08:30:58.743372 dv-cinder-856971-3-iscsi devstack@keystone.service[88051]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 15 08:30:58.743372 dv-cinder-856971-3-iscsi devstack@keystone.service[88051]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 15 08:30:58.743372 dv-cinder-856971-3-iscsi devstack@keystone.service[88051]: nodename: dv-cinder-856971-3-iscsi Oct 15 08:30:58.743372 dv-cinder-856971-3-iscsi devstack@keystone.service[88051]: machine: x86_64 Oct 15 08:30:58.743372 dv-cinder-856971-3-iscsi devstack@keystone.service[88051]: clock source: unix Oct 15 08:30:58.743372 dv-cinder-856971-3-iscsi devstack@keystone.service[88051]: pcre jit disabled Oct 15 08:30:58.743372 dv-cinder-856971-3-iscsi devstack@keystone.service[88051]: detected number of CPU cores: 4 Oct 15 08:30:58.743372 dv-cinder-856971-3-iscsi devstack@keystone.service[88051]: current working directory: / Oct 15 08:30:58.743372 dv-cinder-856971-3-iscsi devstack@keystone.service[88051]: detected binary path: /usr/bin/uwsgi-core Oct 15 08:30:58.743372 dv-cinder-856971-3-iscsi devstack@keystone.service[88051]: your processes number limit is 47680 Oct 15 08:30:58.743372 dv-cinder-856971-3-iscsi devstack@keystone.service[88051]: your memory page size is 4096 bytes Oct 15 08:30:58.743372 dv-cinder-856971-3-iscsi devstack@keystone.service[88051]: detected max file descriptor number: 2048 Oct 15 08:30:58.743372 dv-cinder-856971-3-iscsi devstack@keystone.service[88051]: lock engine: pthread robust mutexes Oct 15 08:30:58.743372 dv-cinder-856971-3-iscsi devstack@keystone.service[88051]: thunder lock: enabled Oct 15 08:30:58.743372 dv-cinder-856971-3-iscsi devstack@keystone.service[88051]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 15 08:30:58.743372 dv-cinder-856971-3-iscsi devstack@keystone.service[88051]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 15 08:30:58.782982 dv-cinder-856971-3-iscsi devstack@keystone.service[88051]: Python main interpreter initialized at 0x55fe8f4189c0 Oct 15 08:30:58.782982 dv-cinder-856971-3-iscsi devstack@keystone.service[88051]: python threads support enabled Oct 15 08:30:58.782982 dv-cinder-856971-3-iscsi devstack@keystone.service[88051]: your server socket listen backlog is limited to 100 connections Oct 15 08:30:58.782982 dv-cinder-856971-3-iscsi devstack@keystone.service[88051]: your mercy for graceful operations on workers is 90 seconds Oct 15 08:30:58.783310 dv-cinder-856971-3-iscsi devstack@keystone.service[88051]: mapped 403077 bytes (393 KB) for 2 cores Oct 15 08:30:58.783389 dv-cinder-856971-3-iscsi devstack@keystone.service[88051]: *** Operational MODE: preforking *** Oct 15 08:30:58.783657 dv-cinder-856971-3-iscsi devstack@keystone.service[88051]: *** uWSGI is running in multiple interpreter mode *** Oct 15 08:30:58.783657 dv-cinder-856971-3-iscsi devstack@keystone.service[88051]: spawned uWSGI master process (pid: 88051) Oct 15 08:30:58.783846 dv-cinder-856971-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 15 08:30:58.789713 dv-cinder-856971-3-iscsi devstack@keystone.service[88051]: spawned uWSGI worker 1 (pid: 88059, cores: 1) Oct 15 08:30:58.789713 dv-cinder-856971-3-iscsi devstack@keystone.service[88051]: spawned uWSGI worker 2 (pid: 88060, cores: 1) Oct 15 08:30:58.789713 dv-cinder-856971-3-iscsi devstack@keystone.service[88051]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 15 08:31:03.048599 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 15 08:31:03.048599 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: from cryptography.utils import int_from_bytes Oct 15 08:31:03.049387 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 15 08:31:03.049387 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: from cryptography.utils import int_from_bytes Oct 15 08:31:03.049872 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 15 08:31:03.049872 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: from cryptography.utils import int_from_bytes Oct 15 08:31:03.050012 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 15 08:31:03.050012 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: from cryptography.utils import int_from_bytes Oct 15 08:31:03.099532 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 15 08:31:03.099724 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 15 08:31:03.099827 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 15 08:31:03.099920 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] command line args: [] {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 15 08:31:03.099999 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 15 08:31:03.100171 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 15 08:31:03.100257 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] command line args: [] {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 15 08:31:03.100338 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] ================================================================================ {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 15 08:31:03.100439 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 15 08:31:03.100598 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] admin_token = **** {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.100747 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ================================================================================ {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 15 08:31:03.100839 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] config_dir = [] {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.101009 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] admin_token = **** {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.101095 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.101196 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] config_source = [] {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.101280 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] config_dir = [] {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.101378 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] debug = True {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.101463 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.101580 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.101869 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] config_source = [] {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.101869 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] debug = True {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.101989 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.102053 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.102143 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] insecure_debug = False {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.102308 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.102395 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.102462 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] insecure_debug = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.102628 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.102718 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.102859 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] list_limit = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.102945 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.103049 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] log_config_append = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.103130 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] list_limit = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.103232 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.103337 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] log_config_append = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.103441 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] log_dir = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.103543 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.103645 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] log_file = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.103747 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] log_dir = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.103747 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] log_file = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.103974 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.104058 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.104182 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.104260 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.104376 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.104452 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.104576 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.104660 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.104735 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.104859 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.104931 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.105014 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.105131 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.105253 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.105336 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.105467 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.105601 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.105805 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.106461 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.106545 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.106667 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.106776 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.106899 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.107011 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.107099 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.107258 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.107341 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] notification_format = cadf {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.107506 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] notification_format = cadf {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.107595 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.107782 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.108041 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.30/identity {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.108229 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] publish_errors = False {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.108351 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.30/identity {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.108438 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] publish_errors = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.108511 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.108659 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.108747 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.108857 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.108942 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.109055 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.109138 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.109307 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.109402 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.109542 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.109627 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] standard_threads = False {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.109803 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] strict_password_check = False {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.109895 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] standard_threads = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.110037 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] strict_password_check = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.110117 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.110216 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] use_eventlog = False {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.110299 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.110397 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] use_journal = False {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.110478 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] use_eventlog = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.110604 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] use_json = False {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.110685 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] use_journal = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.110763 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] use_stderr = False {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.110841 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] use_json = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.110918 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] use_syslog = False {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.110994 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] use_stderr = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.111095 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] use_syslog = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.111180 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] watch_log_file = False {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.111326 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] watch_log_file = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 08:31:03.111405 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.111582 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.111708 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.111822 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.111903 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.112077 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.112165 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.112316 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.112394 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.112555 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.112640 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.112792 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.112870 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.113030 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.113107 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.113265 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.113344 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.113512 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.113598 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.113785 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.113876 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.114024 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.114110 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.114256 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.114336 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.114498 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.114580 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.114734 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.114815 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.114980 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.115060 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.115216 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.115294 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.115452 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.115531 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.115706 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.115785 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.115951 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.116029 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.116180 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.116256 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.116417 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.116496 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.116667 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.116755 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.116917 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.117001 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.117167 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.117246 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.117399 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.117484 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.117632 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.117758 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.117889 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.117967 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.118121 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.118212 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.118358 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.118438 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.118604 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.118682 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.118836 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.118925 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.119065 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.119149 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.119301 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.119380 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.119546 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.119634 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.119781 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.119859 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.120015 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.120094 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.120250 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.120340 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.120480 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.120564 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.120714 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.120793 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.120941 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.121031 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.121169 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.121249 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.121437 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.121519 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.121688 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.121849 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.121926 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.122004 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.122153 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.122233 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.122388 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.122475 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.122620 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.122699 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.122850 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.122928 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.123080 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.123196 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.123316 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.123422 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.123549 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.123662 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.123786 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.123902 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.124002 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.124116 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.124245 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.124366 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.124479 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.124610 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.124711 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.124836 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.124947 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.125027 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.125158 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.125246 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.125390 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.125515 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.125627 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.125735 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.125887 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.125969 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.126120 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.126208 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.126350 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.126438 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.126604 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.126690 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.126837 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.126918 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.127066 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.127148 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.127302 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.127379 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.127535 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.127615 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.127799 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.127880 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.128035 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.128168 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.128301 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.128381 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.128501 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.128577 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.128734 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.128816 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.128968 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.129053 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.129208 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.129284 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] cache.enabled = True {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.129446 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.129532 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.enabled = True {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.129699 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.129783 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.129944 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.130024 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.130187 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.130268 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.130427 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.130510 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.130675 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.130755 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.130913 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.130994 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.131150 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.131231 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.131386 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.131465 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.131621 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.131700 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.131860 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.131947 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.132109 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.132189 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.132347 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.132426 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.132590 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.132678 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.132830 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.132909 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.133069 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.133149 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.133298 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.133374 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.133530 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.133610 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.133790 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.133885 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.134026 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.134116 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.134257 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.134337 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.134487 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.134567 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.134715 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.134803 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.134945 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.135025 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.135172 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.135252 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.135411 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.135499 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.135645 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.135725 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.135886 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.135966 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.136120 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.136207 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.136374 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.136455 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.136617 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.136705 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.136852 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] auth.external = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.136948 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.137115 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] auth.mapped = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.137257 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] auth.external = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.137388 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] auth.mapped = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.137505 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.137649 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.137828 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.137957 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] auth.password = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.138076 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.138194 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] auth.token = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.138303 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] auth.password = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.138408 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] auth.token = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.138515 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.138622 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.138739 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] catalog.caching = True {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.138849 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] catalog.caching = True {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.138963 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.139064 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.139140 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.139268 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.139355 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.139497 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.139587 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.139732 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.139812 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.139963 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.140051 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] credential.caching = True {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.140199 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] credential.caching = True {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.140280 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] credential.driver = sql {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.140443 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.140523 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] credential.driver = sql {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.140688 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.140776 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.140930 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.141010 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.141169 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.141248 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.141403 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.141496 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.141636 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.141738 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.141907 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.142251 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.142401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.142490 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.142643 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.142730 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.142896 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.142978 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.143157 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.143246 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.143405 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.143486 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.143650 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.143729 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.143894 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.143986 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.144134 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] federation.caching = True {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.144213 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.144372 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] federation.caching = True {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.144457 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.144609 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.144698 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] federation.driver = sql {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.144853 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.144933 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] federation.driver = sql {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.145086 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.145173 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.145314 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.145402 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.145560 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.145640 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.145820 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.145902 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.146064 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.146147 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.146299 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.146379 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.146547 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.146627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.146783 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.146864 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.147021 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] identity.caching = True {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.147100 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.147254 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] identity.caching = True {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.147335 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.147503 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.147587 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.147741 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.147822 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.147976 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.148056 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.148212 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.148296 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] identity.driver = sql {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.148443 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.148524 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] identity.driver = sql {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.148676 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.148756 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.148913 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.148995 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.149162 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.149242 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.149395 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.149480 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.149627 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.149827 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.149923 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.150000 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.150110 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.150190 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.150360 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.150471 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.150604 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.150686 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.150849 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.150929 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.151096 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.151177 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.151339 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.151419 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.151595 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.151673 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.151833 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.151914 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.152065 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.152153 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.152296 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.152374 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.152524 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.153086 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.153275 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.153355 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.153515 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.153593 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.153771 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.153853 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.154009 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.154088 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.154242 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.154327 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.154476 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.154557 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.154722 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.154801 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.154955 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.155034 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.155190 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.155269 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.155433 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.155511 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.155664 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.155758 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.155892 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.155970 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.156122 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] ldap.password = **** {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.156199 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.156348 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.password = **** {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.156435 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.156586 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.156670 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.156820 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.156907 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.157053 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.157141 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.157289 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.157409 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.157641 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.157751 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.157923 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.158019 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.158158 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.158237 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.158394 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.158472 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.158627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.158719 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.158877 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.158957 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.159112 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.159190 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.159347 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.159434 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.159584 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.159672 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.159818 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] ldap.user = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.159896 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.160048 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.user = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.160134 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.160288 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.160368 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.160528 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.160606 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.160765 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.160851 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.161019 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.161097 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.161268 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.161347 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.161505 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.161592 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.161759 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.161844 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.162001 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.162081 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.162239 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.162331 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.162478 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.162558 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.162710 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.162789 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.162941 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.163028 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.163189 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.163269 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.163434 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.163511 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.163678 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.163765 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.163922 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.164002 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.164154 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.164234 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.164386 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.164482 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.164628 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.164707 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.164865 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.164950 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.165099 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.165187 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.165333 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.165411 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.165568 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.165647 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.165837 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.165930 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.166077 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.166157 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.166315 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.166398 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] policy.driver = sql {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.166553 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.166640 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] policy.driver = sql {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.166786 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.166865 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.167024 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.167102 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.167258 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.167346 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] receipt.caching = True {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.167494 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.167573 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] receipt.caching = True {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.167729 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.167807 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.167973 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.168060 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.168205 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.168334 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.169014 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.169100 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.169260 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.169349 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] resource.caching = True {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.169503 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] resource.caching = True {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.169582 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.169772 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] resource.driver = sql {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.169856 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.170014 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.170107 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] resource.driver = sql {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.170249 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.170329 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.170496 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.170575 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.170734 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] revoke.caching = True {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.170821 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.170971 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] revoke.caching = True {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.171051 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.171217 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.171296 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.171445 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.171520 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] role.cache_time = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.171679 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] role.cache_time = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.171760 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] role.caching = True {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.171915 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] role.caching = True {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.171995 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] role.driver = sql {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.172145 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] role.list_limit = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.172225 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] role.driver = sql {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.172380 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] role.list_limit = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.172460 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.172627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.172707 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.172875 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.172955 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.173120 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.173199 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.173367 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.173452 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.173611 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.173741 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.173881 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.173962 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.174128 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.174207 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.174363 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.174445 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.174595 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.174674 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.174842 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.174922 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.175087 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.175167 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.175332 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.175411 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.175581 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.175732 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.175969 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.176056 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.176203 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.176282 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.176451 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.176532 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.176699 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.176779 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.176945 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.177023 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.177179 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.177266 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.177413 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.177493 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.177667 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.177779 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.177931 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.178023 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.178166 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.178246 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.178397 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.178476 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.178628 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.178717 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.178860 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.178940 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.179109 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.179188 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.179356 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.179442 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.179596 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.179676 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.179833 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.179913 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.180065 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] token.cache_time = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.180152 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.180303 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] token.caching = True {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.180382 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] token.cache_time = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.180540 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.180619 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] token.caching = True {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.180789 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] token.provider = fernet {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.180878 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.181035 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] token.provider = fernet {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.181113 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.181272 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.181351 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.181524 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.181610 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.181784 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.181866 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.182020 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.182098 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.182263 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.182354 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.182504 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.182584 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] trust.driver = sql {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.182739 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.182819 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] trust.driver = sql {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.182972 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.183059 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.183205 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.183451 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.183529 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.183708 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.183786 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.183959 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.184049 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.184205 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.184286 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.184470 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.184550 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.184715 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.184803 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] database.connection = **** {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.184965 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.185053 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.185214 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.185294 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] database.connection = **** {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.185454 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.185533 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.185722 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.185820 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.185958 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.186040 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.186196 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.186276 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.186464 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.186562 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.186664 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.186744 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.186897 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.186976 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.187134 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.187223 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.187376 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.187455 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.187609 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.187687 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.187851 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.187940 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.188086 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.188166 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.188330 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.188409 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.188560 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.188648 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.188801 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.188881 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.189035 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.189115 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.189208 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 15 08:31:03.189327 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.189562 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 08:31:03.189755 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 15 08:31:03.218727 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=88060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 08:31:03.218856 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=88059) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 08:31:03.218949 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=88060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 08:31:03.219027 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=88059) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 08:31:03.219169 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=88060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 08:31:03.219254 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=88059) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 08:31:03.227411 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=88059) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 08:31:03.227593 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=88060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 08:31:03.255639 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=88060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 08:31:03.255752 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=88059) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 08:31:03.255870 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=88060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 08:31:03.255954 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=88059) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 08:31:03.309667 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=88060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 08:31:03.309855 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=88059) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 08:31:03.358148 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=88060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 08:31:03.358381 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=88060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 08:31:03.359109 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=88059) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 08:31:03.359340 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=88059) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 08:31:03.513104 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 08:31:03.513340 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=88060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 08:31:03.513563 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=88060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 08:31:03.513812 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=88060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 08:31:03.514041 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=88060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 08:31:03.514125 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88059) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 08:31:03.514261 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=88060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 08:31:03.514343 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=88059) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 08:31:03.514481 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=88059) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 08:31:03.514700 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=88059) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 08:31:03.514916 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=88059) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 08:31:03.515129 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=88059) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 08:31:03.537225 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 08:31:03.537465 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=88060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 08:31:03.537575 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88059) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 08:31:03.537700 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=88060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 08:31:03.537818 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=88059) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 08:31:03.537917 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=88060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 08:31:03.538048 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=88059) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 08:31:03.538131 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=88060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 08:31:03.538274 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=88059) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 08:31:03.538357 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=88060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 08:31:03.538496 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=88059) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 08:31:03.538576 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 08:31:03.538712 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=88059) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 08:31:03.538925 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88059) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 08:31:03.550179 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=88060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 08:31:03.550272 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=88059) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 08:31:03.562810 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 08:31:03.563053 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=88060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 08:31:03.563190 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88059) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 08:31:03.563284 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=88060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 08:31:03.563363 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=88059) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 08:31:03.563438 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=88060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 08:31:03.563513 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=88059) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 08:31:03.563631 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=88060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 08:31:03.563727 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=88059) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 08:31:03.563848 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=88060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 08:31:03.563932 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=88059) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 08:31:03.564064 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 08:31:03.564147 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=88059) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 08:31:03.564314 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88059) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 08:31:03.581143 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.581354 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 15 08:31:03.581700 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.581787 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.581913 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 15 08:31:03.582012 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 15 08:31:03.582189 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.582273 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.582441 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 15 08:31:03.582524 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.582699 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.582782 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.582960 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.583042 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.583218 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.583480 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.592599 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.592883 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.593158 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.593447 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.593530 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.593777 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.594053 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.594340 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.600414 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=88060) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 08:31:03.601229 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=88059) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 08:31:03.603339 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=88060) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 08:31:03.603630 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.603916 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.604136 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=88059) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 08:31:03.604231 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.604422 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.604520 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.604705 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.604789 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.604986 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.605076 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.605262 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.605352 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.605523 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.605604 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.606367 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.606474 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.606643 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.606726 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.606916 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.607191 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.607464 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.629265 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.630023 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.630764 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=88060) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 08:31:03.631052 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.631482 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=88059) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 08:31:03.631767 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.635723 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=88060) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 08:31:03.636009 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.636294 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.636376 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=88059) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 08:31:03.636661 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.637457 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.643481 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=88060) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 08:31:03.643772 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.644039 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=88059) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 08:31:03.644435 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.649345 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=88060) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 08:31:03.649641 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.649850 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=88059) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 08:31:03.649957 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.650146 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.650240 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.650435 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.650517 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.650723 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.650805 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.651004 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.651086 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.651279 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.651362 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.651548 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.651815 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.666791 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.667079 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.667216 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.667504 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.671518 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=88060) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 08:31:03.671944 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=88059) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 08:31:03.674778 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.675078 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.675197 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.675493 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.679280 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=88060) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 08:31:03.679672 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=88059) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 08:31:03.682526 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=88060) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 08:31:03.682794 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=88059) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 08:31:03.685822 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.686128 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.686212 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.686419 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.686501 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.686699 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.686788 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.686993 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.687129 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.687253 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.687335 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.687541 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.703529 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=88060) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 08:31:03.703820 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.704057 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=88059) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 08:31:03.704140 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.704341 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.704434 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.704608 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.704875 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.711208 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.711695 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.712754 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.713231 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.714319 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.714600 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.714784 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.715137 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.717453 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=88060) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 08:31:03.717766 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.717849 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=88059) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 08:31:03.718063 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.718145 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.718351 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.718434 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.718666 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.718756 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.719052 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.729856 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=88060) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 08:31:03.730140 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=88059) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 08:31:03.730222 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.730425 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.730506 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.730741 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.730826 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.731031 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.731175 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.731413 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.731503 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.731701 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.731782 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.731981 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.747672 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=88060) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 08:31:03.747836 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=88059) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 08:31:03.750615 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=88060) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 08:31:03.750747 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=88059) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 08:31:03.753484 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.753605 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.755052 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.755137 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.756528 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=88060) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 08:31:03.756701 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=88059) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 08:31:03.756816 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.756994 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.757230 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.757405 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.764173 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.764340 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.765648 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=88060) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 08:31:03.765810 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=88059) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 08:31:03.768830 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.768948 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.769229 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.769344 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.769518 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.769614 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.769824 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.769930 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.777609 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=88060) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 08:31:03.777710 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=88059) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 08:31:03.777935 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.778017 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.778220 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.778302 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.785101 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=88060) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 08:31:03.785205 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=88059) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 08:31:03.785383 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.785465 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.785665 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.785767 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.785967 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.786057 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.786243 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.786325 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.786517 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.786600 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.786792 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.786885 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.787081 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.787210 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.787353 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.787434 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.787629 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.787709 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.787898 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.787986 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.788175 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.788255 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.788440 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.788519 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.788700 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.788789 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=88060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 08:31:03.823066 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55fe8f4189c0 pid: 88060 (default app) Oct 15 08:31:03.823161 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55fe8f4189c0 pid: 88059 (default app) Oct 15 08:31:03.826329 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8485428-33ee-4e04-bca7-3b7bb40fc66d None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:03.826624 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8485428-33ee-4e04-bca7-3b7bb40fc66d None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:03.826901 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8485428-33ee-4e04-bca7-3b7bb40fc66d None None] PATH_INFO: `/v3` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:03.831043 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1/1] 10.222.0.30 () {50 vars in 791 bytes} [Sat Oct 15 08:31:03 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:10.539172 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f3515f-b99b-4648-92d3-045bdae118dc None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:10.539761 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f3515f-b99b-4648-92d3-045bdae118dc None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:10.539761 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f3515f-b99b-4648-92d3-045bdae118dc None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:10.544531 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1/2] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:31:10 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 08:31:10.549778 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c50f94-81b1-4ecb-9704-aee467fd372a None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:10.550243 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c50f94-81b1-4ecb-9704-aee467fd372a None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:10.550391 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c50f94-81b1-4ecb-9704-aee467fd372a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:10.626244 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: INFO dbcounter [None req-08c50f94-81b1-4ecb-9704-aee467fd372a None None] Registered counter for database keystone Oct 15 08:31:10.645647 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG oslo_db.sqlalchemy.engines [None req-08c50f94-81b1-4ecb-9704-aee467fd372a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=88060) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 15 08:31:10.651795 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writer thread running {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 15 08:31:10.830167 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG passlib.handlers.bcrypt [None req-08c50f94-81b1-4ecb-9704-aee467fd372a None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=88060) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 15 08:31:10.830560 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG passlib.handlers.bcrypt [None req-08c50f94-81b1-4ecb-9704-aee467fd372a None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=88060) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 15 08:31:10.888287 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-08c50f94-81b1-4ecb-9704-aee467fd372a None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:31:10.889902 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-08c50f94-81b1-4ecb-9704-aee467fd372a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:31:11.014865 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 2/3] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:31:10 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 466 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 15 08:31:11.463894 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f3515f-b99b-4648-92d3-045bdae118dc None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:11.533328 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: INFO dbcounter [None req-22f3515f-b99b-4648-92d3-045bdae118dc None None] Registered counter for database keystone Oct 15 08:31:11.554241 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG oslo_db.sqlalchemy.engines [None req-22f3515f-b99b-4648-92d3-045bdae118dc None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=88059) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 15 08:31:11.560905 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writer thread running {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 15 08:31:11.679302 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f3515f-b99b-4648-92d3-045bdae118dc None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:11.684070 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103a3b34-4c65-4bef-a9ec-b8fb919307d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:11.684692 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a3b34-4c65-4bef-a9ec-b8fb919307d7 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:11.684974 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a3b34-4c65-4bef-a9ec-b8fb919307d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:11.685269 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a3b34-4c65-4bef-a9ec-b8fb919307d7 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:11.693278 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103a3b34-4c65-4bef-a9ec-b8fb919307d7 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:11.743628 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: WARNING keystone.common.rbac_enforcer.enforcer [None req-103a3b34-4c65-4bef-a9ec-b8fb919307d7 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 15 08:31:11.743966 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103a3b34-4c65-4bef-a9ec-b8fb919307d7 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:11.753702 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 2/4] 10.222.0.30 () {58 vars in 1324 bytes} [Sat Oct 15 08:31:11 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 15 08:31:11.759601 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c50f94-81b1-4ecb-9704-aee467fd372a None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:11.776924 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c50f94-81b1-4ecb-9704-aee467fd372a None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:11.781889 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300232b-a08b-4fc6-9b49-be7aa6772bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:11.782975 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300232b-a08b-4fc6-9b49-be7aa6772bb9 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:11.783260 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300232b-a08b-4fc6-9b49-be7aa6772bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:11.783541 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300232b-a08b-4fc6-9b49-be7aa6772bb9 admin admin] PATH_INFO: `/v3/projects/d200ced7ea7746bab37279a9888da771` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:11.785115 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c300232b-a08b-4fc6-9b49-be7aa6772bb9 admin admin] RBAC: Authorizing `identity:get_project(project_id=d200ced7ea7746bab37279a9888da771)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:11.836770 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c300232b-a08b-4fc6-9b49-be7aa6772bb9 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 15 08:31:11.837122 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c300232b-a08b-4fc6-9b49-be7aa6772bb9 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:11.842142 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 3/5] 10.222.0.30 () {56 vars in 1207 bytes} [Sat Oct 15 08:31:11 2022] GET /identity/v3/projects/d200ced7ea7746bab37279a9888da771 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:13.943407 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ae9246-bf28-44ac-92fb-8bd8e4386f61 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:13.943964 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ae9246-bf28-44ac-92fb-8bd8e4386f61 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:13.943964 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ae9246-bf28-44ac-92fb-8bd8e4386f61 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:13.948045 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 3/6] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:31:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 08:31:13.954016 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19281eb1-d1be-4c54-883a-8aa4052cdbfd None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:13.954403 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19281eb1-d1be-4c54-883a-8aa4052cdbfd None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:13.954570 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19281eb1-d1be-4c54-883a-8aa4052cdbfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:13.974657 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-19281eb1-d1be-4c54-883a-8aa4052cdbfd None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:31:13.976150 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-19281eb1-d1be-4c54-883a-8aa4052cdbfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:31:13.983844 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 4/7] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:31:13 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 15 08:31:14.245124 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ae9246-bf28-44ac-92fb-8bd8e4386f61 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:14.257728 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ae9246-bf28-44ac-92fb-8bd8e4386f61 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:14.262475 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a7ce4a-2e7c-420d-9f57-871b42d68ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:14.263067 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a7ce4a-2e7c-420d-9f57-871b42d68ae9 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:14.263351 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a7ce4a-2e7c-420d-9f57-871b42d68ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:14.263629 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a7ce4a-2e7c-420d-9f57-871b42d68ae9 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:14.271443 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a7ce4a-2e7c-420d-9f57-871b42d68ae9 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:14.273199 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a7ce4a-2e7c-420d-9f57-871b42d68ae9 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:14.281849 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 4/8] 10.222.0.30 () {58 vars in 1324 bytes} [Sat Oct 15 08:31:14 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 08:31:14.286871 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19281eb1-d1be-4c54-883a-8aa4052cdbfd None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:14.295718 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19281eb1-d1be-4c54-883a-8aa4052cdbfd None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:14.300018 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feb132d-8365-4536-b3da-2837fa3d30b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:14.300639 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feb132d-8365-4536-b3da-2837fa3d30b1 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:14.300920 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feb132d-8365-4536-b3da-2837fa3d30b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:14.301194 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feb132d-8365-4536-b3da-2837fa3d30b1 admin admin] PATH_INFO: `/v3/users/6a470f28aa544b3cb42de175f4e32d94` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:14.302508 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feb132d-8365-4536-b3da-2837fa3d30b1 admin admin] RBAC: Authorizing `identity:get_user(user_id=6a470f28aa544b3cb42de175f4e32d94)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:14.304326 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feb132d-8365-4536-b3da-2837fa3d30b1 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:14.305855 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 5/9] 10.222.0.30 () {56 vars in 1198 bytes} [Sat Oct 15 08:31:14 2022] GET /identity/v3/users/6a470f28aa544b3cb42de175f4e32d94 => generated 247 bytes in 19 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:16.486922 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b227b-e505-43a2-a600-3295e3a18f30 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:16.487455 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b227b-e505-43a2-a600-3295e3a18f30 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:16.487455 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b227b-e505-43a2-a600-3295e3a18f30 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:16.488801 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 5/10] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:31:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:31:16.493972 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bf9276-6274-40be-9575-2ab5bedefa2d None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:16.494368 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bf9276-6274-40be-9575-2ab5bedefa2d None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:16.494540 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bf9276-6274-40be-9575-2ab5bedefa2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:16.513728 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-b7bf9276-6274-40be-9575-2ab5bedefa2d None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:31:16.514920 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-b7bf9276-6274-40be-9575-2ab5bedefa2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:31:16.535763 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9105e8ec-b34c-4be7-b668-ae0b2d87b9a0 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:16.536051 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9105e8ec-b34c-4be7-b668-ae0b2d87b9a0 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:16.536329 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9105e8ec-b34c-4be7-b668-ae0b2d87b9a0 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:16.537655 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 6/11] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:31:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 08:31:16.542818 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028cf16a-40a0-4824-8f1c-710a675e4b05 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:16.543104 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028cf16a-40a0-4824-8f1c-710a675e4b05 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:16.543382 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028cf16a-40a0-4824-8f1c-710a675e4b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:16.585118 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG passlib.handlers.bcrypt [None req-028cf16a-40a0-4824-8f1c-710a675e4b05 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=88059) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 15 08:31:16.585510 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG passlib.handlers.bcrypt [None req-028cf16a-40a0-4824-8f1c-710a675e4b05 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=88059) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 15 08:31:16.596877 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 6/12] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:31:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 08:31:16.634712 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-028cf16a-40a0-4824-8f1c-710a675e4b05 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:31:16.636312 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-028cf16a-40a0-4824-8f1c-710a675e4b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:31:16.735130 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 7/13] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:31:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 08:31:16.867794 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bf9276-6274-40be-9575-2ab5bedefa2d None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:16.892132 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:16.972173 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bf9276-6274-40be-9575-2ab5bedefa2d None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:17.004757 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924788d8-e055-4b97-a061-70d14b89ccf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:17.005401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924788d8-e055-4b97-a061-70d14b89ccf8 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:17.005701 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924788d8-e055-4b97-a061-70d14b89ccf8 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:17.005993 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924788d8-e055-4b97-a061-70d14b89ccf8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:17.006474 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028cf16a-40a0-4824-8f1c-710a675e4b05 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:17.018517 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924788d8-e055-4b97-a061-70d14b89ccf8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:31:17.028029 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-924788d8-e055-4b97-a061-70d14b89ccf8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 08:31:17.028029 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:31:17.028029 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:31:17.028029 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:31:17.028029 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:31:17.028029 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:31:17.028029 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:31:17.028029 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:31:17.028029 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:31:17.028029 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:31:17.028029 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:31:17.028029 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:31:17.028029 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:31:17.028029 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:31:17.028029 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:31:17.028029 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:31:17.029031 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:31:17.029031 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:31:17.029031 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:31:17.029031 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:31:17.029031 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:31:17.029031 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:31:17.029031 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:31:17.029031 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:31:17.029031 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:31:17.029031 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:31:17.029031 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:31:17.029031 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:31:17.029031 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:31:17.029031 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:31:17.029031 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:31:17.029031 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:31:17.029904 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:31:17.029904 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:31:17.029904 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:31:17.029904 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:31:17.029904 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:31:17.029904 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:31:17.029904 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:31:17.029904 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:31:17.029904 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:31:17.029904 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:31:17.029904 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 08:31:17.029904 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:31:17.029904 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 7/14] 10.222.0.30 () {56 vars in 1096 bytes} [Sat Oct 15 08:31:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:31:17.034994 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924788d8-e055-4b97-a061-70d14b89ccf8 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:17.109576 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028cf16a-40a0-4824-8f1c-710a675e4b05 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:17.111833 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:17.115908 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:17.134881 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924788d8-e055-4b97-a061-70d14b89ccf8 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:17.148362 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8596b1-9f25-48bb-b3de-b1433eb25e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:17.149215 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8596b1-9f25-48bb-b3de-b1433eb25e4b None admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:17.149498 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8596b1-9f25-48bb-b3de-b1433eb25e4b None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:17.149800 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8596b1-9f25-48bb-b3de-b1433eb25e4b None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:17.150764 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8596b1-9f25-48bb-b3de-b1433eb25e4b None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:17.152741 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8596b1-9f25-48bb-b3de-b1433eb25e4b None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:17.163579 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.sql.core [None req-4a8596b1-9f25-48bb-b3de-b1433eb25e4b None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 15 08:31:17.163579 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 15 08:31:17.163579 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [parameters: {'id': '24f91e183d0249818803483a12e07881', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 15 08:31:17.163579 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88059) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 15 08:31:17.169857 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7a9c0d-5546-4ae2-9fc0-708c1a3bbfce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:17.170468 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a9c0d-5546-4ae2-9fc0-708c1a3bbfce None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:17.170751 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a9c0d-5546-4ae2-9fc0-708c1a3bbfce None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:17.171026 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a9c0d-5546-4ae2-9fc0-708c1a3bbfce None admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:17.171701 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7a9c0d-5546-4ae2-9fc0-708c1a3bbfce None admin] RBAC: Adding query filter params (name=admin) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:31:17.172010 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7a9c0d-5546-4ae2-9fc0-708c1a3bbfce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:17.173397 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7a9c0d-5546-4ae2-9fc0-708c1a3bbfce None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:17.181873 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-4a8596b1-9f25-48bb-b3de-b1433eb25e4b None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 15 08:31:17.181873 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:31:17.181873 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 15 08:31:17.181873 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 15 08:31:17.181873 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 15 08:31:17.181873 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 15 08:31:17.181873 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 15 08:31:17.181873 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application result = self._query(query) Oct 15 08:31:17.181873 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 15 08:31:17.181873 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application conn.query(q) Oct 15 08:31:17.181873 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 15 08:31:17.181873 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 15 08:31:17.181873 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 15 08:31:17.181873 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application result.read() Oct 15 08:31:17.182673 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 15 08:31:17.182673 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 15 08:31:17.182673 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 15 08:31:17.182673 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 15 08:31:17.182673 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 15 08:31:17.182673 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 15 08:31:17.182673 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 15 08:31:17.182673 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 15 08:31:17.182673 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 15 08:31:17.182673 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:31:17.182673 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 15 08:31:17.182673 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:31:17.182673 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:31:17.182673 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 15 08:31:17.182673 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 15 08:31:17.182673 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 15 08:31:17.182673 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 15 08:31:17.182673 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 15 08:31:17.183717 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application next(self.gen) Oct 15 08:31:17.183717 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 15 08:31:17.183717 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application yield resource Oct 15 08:31:17.183717 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 15 08:31:17.183717 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application next(self.gen) Oct 15 08:31:17.183717 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 15 08:31:17.183717 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self.session.rollback() Oct 15 08:31:17.183717 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 08:31:17.183717 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self.force_reraise() Oct 15 08:31:17.183717 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 08:31:17.183717 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise self.value Oct 15 08:31:17.183717 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 15 08:31:17.183717 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 15 08:31:17.183717 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 15 08:31:17.183717 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application session.commit() Oct 15 08:31:17.183717 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 15 08:31:17.183717 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 15 08:31:17.184600 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 15 08:31:17.184600 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._prepare_impl() Oct 15 08:31:17.184600 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 15 08:31:17.184600 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self.session.flush() Oct 15 08:31:17.184600 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 15 08:31:17.184600 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._flush(objects) Oct 15 08:31:17.184600 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 15 08:31:17.184600 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 15 08:31:17.184600 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 15 08:31:17.184600 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application compat.raise_( Oct 15 08:31:17.184600 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 15 08:31:17.184600 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception Oct 15 08:31:17.184600 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 15 08:31:17.184600 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application flush_context.execute() Oct 15 08:31:17.184600 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 15 08:31:17.184600 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rec.execute(self) Oct 15 08:31:17.184600 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 15 08:31:17.185470 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 15 08:31:17.185470 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 15 08:31:17.185470 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 15 08:31:17.185470 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 15 08:31:17.185470 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 15 08:31:17.185470 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 15 08:31:17.185470 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 15 08:31:17.185470 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 15 08:31:17.185470 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 15 08:31:17.185470 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 15 08:31:17.185470 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 15 08:31:17.185470 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 15 08:31:17.185470 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 15 08:31:17.185470 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 15 08:31:17.185470 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 15 08:31:17.187164 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 15 08:31:17.187164 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception Oct 15 08:31:17.187164 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 15 08:31:17.187164 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 15 08:31:17.187164 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 15 08:31:17.187164 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 15 08:31:17.187164 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 15 08:31:17.187164 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application result = self._query(query) Oct 15 08:31:17.187164 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 15 08:31:17.187164 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application conn.query(q) Oct 15 08:31:17.187164 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 15 08:31:17.187164 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 15 08:31:17.187164 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 15 08:31:17.187164 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application result.read() Oct 15 08:31:17.187164 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 15 08:31:17.187164 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 15 08:31:17.187164 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 15 08:31:17.188434 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 15 08:31:17.188434 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 15 08:31:17.188434 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 15 08:31:17.188434 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 15 08:31:17.188434 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 15 08:31:17.188434 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 15 08:31:17.188434 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 15 08:31:17.188434 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [parameters: {'id': '24f91e183d0249818803483a12e07881', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 15 08:31:17.188434 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 15 08:31:17.188434 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:31:17.188434 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 08:31:17.188434 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:31:17.188434 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:31:17.188434 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 15 08:31:17.188434 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 15 08:31:17.189399 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 15 08:31:17.189399 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 15 08:31:17.189399 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 15 08:31:17.189399 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:31:17.189399 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 08:31:17.189399 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:31:17.189399 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:31:17.189399 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:31:17.189399 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:31:17.189399 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:31:17.189399 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:31:17.189399 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:31:17.189399 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:31:17.189399 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:31:17.189399 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:31:17.189399 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:31:17.189399 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:31:17.190737 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 15 08:31:17.190737 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 15 08:31:17.190737 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:31:17.190737 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:31:17.190737 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 15 08:31:17.190737 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 15 08:31:17.190737 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:31:17.190737 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:31:17.190737 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 15 08:31:17.190737 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 15 08:31:17.190737 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 15 08:31:17.190737 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:31:17.190737 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 8/15] 10.222.0.30 () {60 vars in 1137 bytes} [Sat Oct 15 08:31:17 2022] POST /identity/v3/domains => generated 190 bytes in 177 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 15 08:31:17.191556 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 8/16] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 08:31:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 08:31:17.191635 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8596b1-9f25-48bb-b3de-b1433eb25e4b None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:17.191721 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7a9c0d-5546-4ae2-9fc0-708c1a3bbfce None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:17.289253 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8596b1-9f25-48bb-b3de-b1433eb25e4b None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:17.289572 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7a9c0d-5546-4ae2-9fc0-708c1a3bbfce None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:17.321763 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d2fa59-caa2-4dc1-a6ea-b075c87ce7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:17.322655 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d2fa59-caa2-4dc1-a6ea-b075c87ce7b0 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:17.322961 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d2fa59-caa2-4dc1-a6ea-b075c87ce7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:17.323077 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d9f91-808d-4f14-9d16-9f853d5e050e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:17.323240 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d2fa59-caa2-4dc1-a6ea-b075c87ce7b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:17.323701 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d9f91-808d-4f14-9d16-9f853d5e050e None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:17.324557 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d9f91-808d-4f14-9d16-9f853d5e050e None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:17.324820 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d9f91-808d-4f14-9d16-9f853d5e050e None admin] PATH_INFO: `/v3/users/6a470f28aa544b3cb42de175f4e32d94` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:17.325985 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7d9f91-808d-4f14-9d16-9f853d5e050e None admin] RBAC: Authorizing `identity:get_user(user_id=6a470f28aa544b3cb42de175f4e32d94)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:17.327445 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7d9f91-808d-4f14-9d16-9f853d5e050e None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:17.329012 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 9/17] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 08:31:17 2022] GET /identity/v3/users/6a470f28aa544b3cb42de175f4e32d94 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 08:31:17.334543 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d9f91-808d-4f14-9d16-9f853d5e050e None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:17.341900 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d2fa59-caa2-4dc1-a6ea-b075c87ce7b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:17.343349 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d2fa59-caa2-4dc1-a6ea-b075c87ce7b0 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:17.348995 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:17.359228 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-33d2fa59-caa2-4dc1-a6ea-b075c87ce7b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:31:17.359228 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:31:17.359228 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:31:17.359228 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:31:17.359228 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:31:17.359228 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:31:17.359228 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:31:17.359228 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:31:17.359228 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:31:17.359228 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:31:17.359228 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:31:17.359228 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:31:17.359228 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 08:31:17.359228 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 08:31:17.359228 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 08:31:17.359228 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 08:31:17.360152 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:31:17.360152 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:31:17.360152 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:31:17.360152 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:31:17.360152 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:31:17.360152 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:31:17.360152 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:31:17.360152 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:31:17.360152 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:31:17.360152 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:31:17.360152 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:31:17.360152 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:31:17.360152 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:31:17.360152 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:31:17.360152 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:31:17.360152 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:31:17.360152 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 08:31:17.360992 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 08:31:17.360992 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:31:17.360992 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:31:17.360992 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 9/18] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:31:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 08:31:17.365950 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d2fa59-caa2-4dc1-a6ea-b075c87ce7b0 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:17.387836 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:17.441068 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d9f91-808d-4f14-9d16-9f853d5e050e None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:17.470163 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d2fa59-caa2-4dc1-a6ea-b075c87ce7b0 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:17.475855 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b2a7f5-b782-4f56-9c50-838033ecc094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:17.476478 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b2a7f5-b782-4f56-9c50-838033ecc094 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:17.476759 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b2a7f5-b782-4f56-9c50-838033ecc094 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:17.477033 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b2a7f5-b782-4f56-9c50-838033ecc094 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:17.478160 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b2a7f5-b782-4f56-9c50-838033ecc094 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:17.479611 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b2a7f5-b782-4f56-9c50-838033ecc094 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:17.481891 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 10/19] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:31:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:17.487406 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b2a7f5-b782-4f56-9c50-838033ecc094 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:17.503478 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc35de4c-40e1-4596-b074-b77a3f859031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:17.504074 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc35de4c-40e1-4596-b074-b77a3f859031 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:17.504359 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc35de4c-40e1-4596-b074-b77a3f859031 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:17.504633 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc35de4c-40e1-4596-b074-b77a3f859031 None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:17.505271 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc35de4c-40e1-4596-b074-b77a3f859031 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:31:17.505571 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc35de4c-40e1-4596-b074-b77a3f859031 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:17.506964 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc35de4c-40e1-4596-b074-b77a3f859031 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:17.523957 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 10/20] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 08:31:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 08:31:17.573144 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:17.594104 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b2a7f5-b782-4f56-9c50-838033ecc094 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:17.630085 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e29cb56-9d01-4bd8-993e-aa76a026559e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:17.630763 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e29cb56-9d01-4bd8-993e-aa76a026559e None admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:17.631043 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e29cb56-9d01-4bd8-993e-aa76a026559e None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:17.631320 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e29cb56-9d01-4bd8-993e-aa76a026559e None admin] PATH_INFO: `/v3/domains/default/users/6a470f28aa544b3cb42de175f4e32d94/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:17.633441 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e29cb56-9d01-4bd8-993e-aa76a026559e None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=6a470f28aa544b3cb42de175f4e32d94, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:17.634989 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e29cb56-9d01-4bd8-993e-aa76a026559e None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:17.751494 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 11/21] 10.222.0.30 () {58 vars in 1361 bytes} [Sat Oct 15 08:31:17 2022] PUT /identity/v3/domains/default/users/6a470f28aa544b3cb42de175f4e32d94/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 08:31:19.933223 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ad9a5-ed89-46a1-bfac-315153b029b7 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:19.934747 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ad9a5-ed89-46a1-bfac-315153b029b7 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:19.935527 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ad9a5-ed89-46a1-bfac-315153b029b7 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:19.938287 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 11/22] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:31:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:31:19.946467 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f2da17-c2aa-4924-a887-4f4e6aac1e40 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:19.947166 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f2da17-c2aa-4924-a887-4f4e6aac1e40 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:19.947166 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f2da17-c2aa-4924-a887-4f4e6aac1e40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:19.974807 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-66f2da17-c2aa-4924-a887-4f4e6aac1e40 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:31:19.976363 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-66f2da17-c2aa-4924-a887-4f4e6aac1e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:31:20.066270 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 12/23] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:31:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 08:31:20.469045 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d149d-a941-4a08-b15f-456c1a05aeee None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:20.469330 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d149d-a941-4a08-b15f-456c1a05aeee None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:20.469879 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d149d-a941-4a08-b15f-456c1a05aeee None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:20.471752 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 12/24] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:31:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 08:31:20.477217 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7909fe-2d04-4316-a497-22272eb49f6f None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:20.477762 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7909fe-2d04-4316-a497-22272eb49f6f None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:20.478251 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7909fe-2d04-4316-a497-22272eb49f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:20.507758 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-6d7909fe-2d04-4316-a497-22272eb49f6f None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:31:20.509312 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-6d7909fe-2d04-4316-a497-22272eb49f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:31:20.602073 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:20.607564 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 13/25] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:31:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 08:31:20.791610 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d149d-a941-4a08-b15f-456c1a05aeee None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:20.883164 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:20.905393 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d149d-a941-4a08-b15f-456c1a05aeee None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:20.940574 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8670639-4e65-4757-8ecb-1766de825d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:20.941968 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8670639-4e65-4757-8ecb-1766de825d18 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:20.942488 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8670639-4e65-4757-8ecb-1766de825d18 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:20.942974 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8670639-4e65-4757-8ecb-1766de825d18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:20.954002 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8670639-4e65-4757-8ecb-1766de825d18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:20.955706 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8670639-4e65-4757-8ecb-1766de825d18 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:20.970888 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-e8670639-4e65-4757-8ecb-1766de825d18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:31:20.970888 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:31:20.970888 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:31:20.970888 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:31:20.970888 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:31:20.970888 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:31:20.970888 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:31:20.970888 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:31:20.970888 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:31:20.970888 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:31:20.970888 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:31:20.970888 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:31:20.970888 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 08:31:20.970888 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 08:31:20.970888 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 08:31:20.970888 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 08:31:20.971744 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:31:20.971744 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:31:20.971744 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:31:20.971744 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:31:20.971744 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:31:20.971744 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:31:20.971744 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:31:20.971744 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:31:20.971744 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:31:20.971744 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:31:20.971744 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:31:20.971744 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:31:20.971744 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:31:20.971744 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:31:20.971744 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:31:20.971744 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:31:20.971744 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 08:31:20.972529 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 08:31:20.972529 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:31:20.972529 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:31:20.974596 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 13/26] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:31:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:31:20.982763 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7909fe-2d04-4316-a497-22272eb49f6f None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:21.018955 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5bc9e-aaee-4378-b55a-29b9f7fdc260 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:21.019635 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5bc9e-aaee-4378-b55a-29b9f7fdc260 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:21.020120 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5bc9e-aaee-4378-b55a-29b9f7fdc260 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:21.021947 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 14/27] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:31:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:31:21.029256 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9e8c8a-fbf1-43cc-824d-eec54dd18328 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:21.029822 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9e8c8a-fbf1-43cc-824d-eec54dd18328 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:21.030306 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9e8c8a-fbf1-43cc-824d-eec54dd18328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:21.060147 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-cf9e8c8a-fbf1-43cc-824d-eec54dd18328 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:31:21.062456 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-cf9e8c8a-fbf1-43cc-824d-eec54dd18328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:31:21.137633 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 15/28] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:31:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 08:31:21.181850 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9e8c8a-fbf1-43cc-824d-eec54dd18328 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:21.202075 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7909fe-2d04-4316-a497-22272eb49f6f None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:21.302310 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:21.318491 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da2009b-4ca7-4d21-97a2-137f701a1969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:21.319047 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da2009b-4ca7-4d21-97a2-137f701a1969 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:21.319340 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da2009b-4ca7-4d21-97a2-137f701a1969 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:21.319621 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da2009b-4ca7-4d21-97a2-137f701a1969 None admin] PATH_INFO: `/v3/domains` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:21.320297 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da2009b-4ca7-4d21-97a2-137f701a1969 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:31:21.320609 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da2009b-4ca7-4d21-97a2-137f701a1969 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:21.322294 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da2009b-4ca7-4d21-97a2-137f701a1969 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:21.368095 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 14/29] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 08:31:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 386 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:21.374652 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da2009b-4ca7-4d21-97a2-137f701a1969 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:21.381157 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9e8c8a-fbf1-43cc-824d-eec54dd18328 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:21.506637 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ad2d8e-6f07-4526-8f8b-af35e07b01e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:21.507606 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ad2d8e-6f07-4526-8f8b-af35e07b01e7 None admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:21.508017 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ad2d8e-6f07-4526-8f8b-af35e07b01e7 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:21.508165 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:21.508494 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ad2d8e-6f07-4526-8f8b-af35e07b01e7 None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:21.509664 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ad2d8e-6f07-4526-8f8b-af35e07b01e7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:21.512658 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ad2d8e-6f07-4526-8f8b-af35e07b01e7 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:21.601884 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da2009b-4ca7-4d21-97a2-137f701a1969 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:21.679638 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec12ac4-e722-44c6-b849-d3043cd50225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:21.680648 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec12ac4-e722-44c6-b849-d3043cd50225 None admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:21.681151 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec12ac4-e722-44c6-b849-d3043cd50225 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:21.681651 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec12ac4-e722-44c6-b849-d3043cd50225 None admin] PATH_INFO: `/v3/projects` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:21.683015 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec12ac4-e722-44c6-b849-d3043cd50225 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:21.685783 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec12ac4-e722-44c6-b849-d3043cd50225 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:21.704859 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 16/30] 10.222.0.30 () {60 vars in 1131 bytes} [Sat Oct 15 08:31:21 2022] POST /identity/v3/roles => generated 219 bytes in 524 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:31:21.706454 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacb685c-4636-49aa-893b-7620d3509a37 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:21.706753 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacb685c-4636-49aa-893b-7620d3509a37 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:21.707119 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacb685c-4636-49aa-893b-7620d3509a37 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:21.708332 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 17/31] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:31:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:31:21.713351 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacb685c-4636-49aa-893b-7620d3509a37 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:21.753221 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 15/32] 10.222.0.30 () {60 vars in 1141 bytes} [Sat Oct 15 08:31:21 2022] POST /identity/v3/projects => generated 301 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 08:31:21.754810 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d824c6f4-adae-4a56-964f-20fcf47529f8 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:21.755108 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d824c6f4-adae-4a56-964f-20fcf47529f8 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:21.755396 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d824c6f4-adae-4a56-964f-20fcf47529f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:21.790962 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-d824c6f4-adae-4a56-964f-20fcf47529f8 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:31:21.792772 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-d824c6f4-adae-4a56-964f-20fcf47529f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:31:21.928504 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacb685c-4636-49aa-893b-7620d3509a37 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:21.957999 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 16/33] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:31:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 08:31:21.962045 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:21.979585 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4f41eb-0554-4fee-90ec-e653da5e9d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:21.980412 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4f41eb-0554-4fee-90ec-e653da5e9d69 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:21.980690 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4f41eb-0554-4fee-90ec-e653da5e9d69 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:21.980975 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4f41eb-0554-4fee-90ec-e653da5e9d69 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:21.996989 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4f41eb-0554-4fee-90ec-e653da5e9d69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:31:22.003679 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-5e4f41eb-0554-4fee-90ec-e653da5e9d69 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 08:31:22.003679 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:31:22.003679 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:31:22.003679 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:31:22.003679 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:31:22.003679 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:31:22.003679 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:31:22.003679 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:31:22.003679 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:31:22.003679 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:31:22.003679 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:31:22.003679 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:31:22.003679 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:31:22.003679 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:31:22.003679 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:31:22.003679 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:31:22.004513 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:31:22.004513 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:31:22.004513 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:31:22.004513 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:31:22.004513 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:31:22.004513 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:31:22.004513 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:31:22.004513 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:31:22.004513 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:31:22.004513 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:31:22.004513 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:31:22.004513 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:31:22.004513 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:31:22.004513 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:31:22.004513 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:31:22.004513 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:31:22.005249 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:31:22.005249 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:31:22.005249 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:31:22.005249 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:31:22.005249 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:31:22.005249 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:31:22.005249 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:31:22.005249 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:31:22.005249 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:31:22.005249 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:31:22.005249 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 08:31:22.005249 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:31:22.005249 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 18/34] 10.222.0.30 () {56 vars in 1096 bytes} [Sat Oct 15 08:31:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:31:22.010993 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d824c6f4-adae-4a56-964f-20fcf47529f8 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:22.069916 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:22.146114 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d824c6f4-adae-4a56-964f-20fcf47529f8 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:22.185589 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c260b25-0aa7-4b61-b04d-ad3c1fc767e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:22.186566 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c260b25-0aa7-4b61-b04d-ad3c1fc767e6 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:22.187049 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c260b25-0aa7-4b61-b04d-ad3c1fc767e6 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:22.187522 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c260b25-0aa7-4b61-b04d-ad3c1fc767e6 None admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:22.188386 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c260b25-0aa7-4b61-b04d-ad3c1fc767e6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:31:22.188905 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c260b25-0aa7-4b61-b04d-ad3c1fc767e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:22.190565 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c260b25-0aa7-4b61-b04d-ad3c1fc767e6 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:22.201647 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 17/35] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 08:31:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 08:31:22.207294 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4f41eb-0554-4fee-90ec-e653da5e9d69 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:22.282170 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c260b25-0aa7-4b61-b04d-ad3c1fc767e6 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:22.329632 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4f41eb-0554-4fee-90ec-e653da5e9d69 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:22.369820 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941453f6-4ed4-4c6c-8b5d-74b1873b7a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:22.370554 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941453f6-4ed4-4c6c-8b5d-74b1873b7a29 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:22.370934 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941453f6-4ed4-4c6c-8b5d-74b1873b7a29 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:22.371219 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941453f6-4ed4-4c6c-8b5d-74b1873b7a29 None admin] PATH_INFO: `/v3/users/6a470f28aa544b3cb42de175f4e32d94` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:22.372739 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941453f6-4ed4-4c6c-8b5d-74b1873b7a29 None admin] RBAC: Authorizing `identity:get_user(user_id=6a470f28aa544b3cb42de175f4e32d94)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:22.374462 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941453f6-4ed4-4c6c-8b5d-74b1873b7a29 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:22.376241 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 19/36] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 08:31:22 2022] GET /identity/v3/users/6a470f28aa544b3cb42de175f4e32d94 => generated 247 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:22.381811 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941453f6-4ed4-4c6c-8b5d-74b1873b7a29 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:22.414810 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c260b25-0aa7-4b61-b04d-ad3c1fc767e6 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:22.418894 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:22.449549 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:22.456793 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2b58d7-db30-4df7-9d3c-50ae1133bf35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:22.457884 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2b58d7-db30-4df7-9d3c-50ae1133bf35 None admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:22.458307 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2b58d7-db30-4df7-9d3c-50ae1133bf35 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:22.458592 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2b58d7-db30-4df7-9d3c-50ae1133bf35 None admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:22.459439 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2b58d7-db30-4df7-9d3c-50ae1133bf35 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:22.461455 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2b58d7-db30-4df7-9d3c-50ae1133bf35 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:22.478538 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 18/37] 10.222.0.30 () {60 vars in 1131 bytes} [Sat Oct 15 08:31:22 2022] POST /identity/v3/roles => generated 225 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 08:31:22.478538 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8890132-bc44-4abc-a628-818b5311a9e6 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:22.478538 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8890132-bc44-4abc-a628-818b5311a9e6 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:22.478538 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8890132-bc44-4abc-a628-818b5311a9e6 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:22.478538 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 19/38] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:31:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:31:22.488304 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349e0e22-681a-4718-bc3b-29bf6173dc18 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:22.488304 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349e0e22-681a-4718-bc3b-29bf6173dc18 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:22.488588 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349e0e22-681a-4718-bc3b-29bf6173dc18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:22.510761 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-349e0e22-681a-4718-bc3b-29bf6173dc18 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:31:22.512403 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-349e0e22-681a-4718-bc3b-29bf6173dc18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:31:22.524041 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941453f6-4ed4-4c6c-8b5d-74b1873b7a29 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:22.575531 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926faeaa-d419-4cb2-ac09-f1fd023188d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:22.576315 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926faeaa-d419-4cb2-ac09-f1fd023188d1 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:22.576605 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926faeaa-d419-4cb2-ac09-f1fd023188d1 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:22.576888 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926faeaa-d419-4cb2-ac09-f1fd023188d1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:22.578279 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926faeaa-d419-4cb2-ac09-f1fd023188d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:22.579771 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926faeaa-d419-4cb2-ac09-f1fd023188d1 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:22.585749 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 20/39] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:31:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 08:31:22.585749 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726a0627-9333-40ba-a94a-b07c2d4f7baa None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:22.585749 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726a0627-9333-40ba-a94a-b07c2d4f7baa None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:22.585749 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726a0627-9333-40ba-a94a-b07c2d4f7baa None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:22.585749 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 21/40] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:31:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:31:22.588351 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726a0627-9333-40ba-a94a-b07c2d4f7baa None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:22.600663 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 20/41] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:31:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 08:31:22.602017 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b4d118-7160-4ee8-951b-9c3c7d366338 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:22.602423 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b4d118-7160-4ee8-951b-9c3c7d366338 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:22.602688 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b4d118-7160-4ee8-951b-9c3c7d366338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:22.721166 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726a0627-9333-40ba-a94a-b07c2d4f7baa None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:22.742068 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:22.746751 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-d6b4d118-7160-4ee8-951b-9c3c7d366338 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:31:22.748093 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-d6b4d118-7160-4ee8-951b-9c3c7d366338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:31:22.764788 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb25f5d-87d2-4d7b-b191-9483d896f04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:22.765411 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb25f5d-87d2-4d7b-b191-9483d896f04b None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:22.765713 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb25f5d-87d2-4d7b-b191-9483d896f04b None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:22.765991 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb25f5d-87d2-4d7b-b191-9483d896f04b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:22.766962 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb25f5d-87d2-4d7b-b191-9483d896f04b None admin] RBAC: Adding query filter params (role.id=14615e30ed3f4d8cb78ee5e0bf9b5761, scope.domain.id=default, user.id=6a470f28aa544b3cb42de175f4e32d94) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:31:22.767470 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb25f5d-87d2-4d7b-b191-9483d896f04b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:22.769171 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb25f5d-87d2-4d7b-b191-9483d896f04b None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:22.785714 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 22/42] 10.222.0.30 () {56 vars in 1322 bytes} [Sat Oct 15 08:31:22 2022] GET /identity/v3/role_assignments?user.id=6a470f28aa544b3cb42de175f4e32d94&scope.domain.id=default&role.id=14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 530 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:22.829865 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 21/43] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:31:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 08:31:22.878851 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb25f5d-87d2-4d7b-b191-9483d896f04b None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:22.990252 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb25f5d-87d2-4d7b-b191-9483d896f04b None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:23.027727 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ddefca-f9ca-424c-98c9-e5259658d69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:23.028453 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ddefca-f9ca-424c-98c9-e5259658d69b None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:23.028678 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ddefca-f9ca-424c-98c9-e5259658d69b None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:23.029020 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ddefca-f9ca-424c-98c9-e5259658d69b None admin] PATH_INFO: `/v3/domains/default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:23.030237 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ddefca-f9ca-424c-98c9-e5259658d69b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:23.031618 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ddefca-f9ca-424c-98c9-e5259658d69b None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:23.033197 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 23/44] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:31:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:23.038749 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b4d118-7160-4ee8-951b-9c3c7d366338 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:23.064830 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:23.095877 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ddefca-f9ca-424c-98c9-e5259658d69b None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:23.139403 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b4d118-7160-4ee8-951b-9c3c7d366338 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:23.146994 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:23.174696 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bbcf59-4687-44e0-aa42-0db87547a587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:23.175307 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bbcf59-4687-44e0-aa42-0db87547a587 None admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:23.175602 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bbcf59-4687-44e0-aa42-0db87547a587 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:23.175877 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bbcf59-4687-44e0-aa42-0db87547a587 None admin] PATH_INFO: `/v3/projects` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:23.176526 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bbcf59-4687-44e0-aa42-0db87547a587 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:23.178009 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bbcf59-4687-44e0-aa42-0db87547a587 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:23.196636 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ddefca-f9ca-424c-98c9-e5259658d69b None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:23.204382 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 22/45] 10.222.0.30 () {60 vars in 1141 bytes} [Sat Oct 15 08:31:23 2022] POST /identity/v3/projects => generated 312 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 08:31:23.232450 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35630d4f-766b-4819-a249-cc05431c6dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:23.233097 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35630d4f-766b-4819-a249-cc05431c6dfd None admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:23.233383 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35630d4f-766b-4819-a249-cc05431c6dfd None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:23.233820 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35630d4f-766b-4819-a249-cc05431c6dfd None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:23.234468 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35630d4f-766b-4819-a249-cc05431c6dfd None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:23.235900 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35630d4f-766b-4819-a249-cc05431c6dfd None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:23.252220 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 24/46] 10.222.0.30 () {60 vars in 1131 bytes} [Sat Oct 15 08:31:23 2022] POST /identity/v3/roles => generated 223 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 08:31:25.347998 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2267cea-5b29-481a-94b9-5406020d9c02 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:25.348551 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2267cea-5b29-481a-94b9-5406020d9c02 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:25.348551 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2267cea-5b29-481a-94b9-5406020d9c02 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:25.349962 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 23/47] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:31:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:31:25.355579 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692f880-0cfc-4123-ace6-e7c7df93ef6c None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:25.355988 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692f880-0cfc-4123-ace6-e7c7df93ef6c None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:25.356139 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692f880-0cfc-4123-ace6-e7c7df93ef6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:25.374771 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-a692f880-0cfc-4123-ace6-e7c7df93ef6c None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:31:25.376287 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-a692f880-0cfc-4123-ace6-e7c7df93ef6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:31:25.441341 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 25/48] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:31:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 08:31:25.707687 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2267cea-5b29-481a-94b9-5406020d9c02 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:25.784685 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:25.808746 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2267cea-5b29-481a-94b9-5406020d9c02 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:25.841600 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9e7758-fe46-46e2-a124-d9329747b505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:25.842245 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e7758-fe46-46e2-a124-d9329747b505 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:25.842531 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e7758-fe46-46e2-a124-d9329747b505 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:25.842811 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e7758-fe46-46e2-a124-d9329747b505 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:25.843921 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9e7758-fe46-46e2-a124-d9329747b505 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:25.845368 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9e7758-fe46-46e2-a124-d9329747b505 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:25.846943 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 24/49] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:31:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:25.852421 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a692f880-0cfc-4123-ace6-e7c7df93ef6c None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:25.895167 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:25.951956 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a692f880-0cfc-4123-ace6-e7c7df93ef6c None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:26.089786 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e80ece2-8144-4096-ac3a-6a5c29f9f8e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:26.090398 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e80ece2-8144-4096-ac3a-6a5c29f9f8e8 None admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:26.090684 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e80ece2-8144-4096-ac3a-6a5c29f9f8e8 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:26.090965 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e80ece2-8144-4096-ac3a-6a5c29f9f8e8 None admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:26.091630 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e80ece2-8144-4096-ac3a-6a5c29f9f8e8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:26.093064 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e80ece2-8144-4096-ac3a-6a5c29f9f8e8 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:26.118879 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 26/50] 10.222.0.30 () {60 vars in 1140 bytes} [Sat Oct 15 08:31:25 2022] POST /identity/v3/projects => generated 298 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 08:31:28.235895 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f21346-259e-4069-a37b-793bc449aea1 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:28.236410 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f21346-259e-4069-a37b-793bc449aea1 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:28.236518 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f21346-259e-4069-a37b-793bc449aea1 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:28.237847 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 25/51] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:31:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:31:28.243219 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c564a4-30e6-4cd9-8a3e-b6e755197dbd None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:28.243632 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c564a4-30e6-4cd9-8a3e-b6e755197dbd None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:28.243782 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c564a4-30e6-4cd9-8a3e-b6e755197dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:28.261924 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-36c564a4-30e6-4cd9-8a3e-b6e755197dbd None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:31:28.263117 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-36c564a4-30e6-4cd9-8a3e-b6e755197dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:31:28.326567 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 27/52] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:31:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 08:31:28.600930 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f21346-259e-4069-a37b-793bc449aea1 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:28.699360 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f21346-259e-4069-a37b-793bc449aea1 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:28.732225 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115d2395-bd3f-4886-884f-78bfd68473fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:28.732841 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d2395-bd3f-4886-884f-78bfd68473fc None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:28.733127 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d2395-bd3f-4886-884f-78bfd68473fc None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:28.733407 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d2395-bd3f-4886-884f-78bfd68473fc None admin] PATH_INFO: `/v3/domains/default` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:28.734541 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115d2395-bd3f-4886-884f-78bfd68473fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:28.735990 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115d2395-bd3f-4886-884f-78bfd68473fc None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:28.737536 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 26/53] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:31:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:28.743142 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c564a4-30e6-4cd9-8a3e-b6e755197dbd None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:28.766482 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:28.840157 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c564a4-30e6-4cd9-8a3e-b6e755197dbd None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:28.871745 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de22f9a2-95d5-4ec8-9693-c09c7cce53f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:28.872293 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de22f9a2-95d5-4ec8-9693-c09c7cce53f2 None admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:28.872580 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de22f9a2-95d5-4ec8-9693-c09c7cce53f2 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:28.872861 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de22f9a2-95d5-4ec8-9693-c09c7cce53f2 None admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:28.873504 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de22f9a2-95d5-4ec8-9693-c09c7cce53f2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:28.874954 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de22f9a2-95d5-4ec8-9693-c09c7cce53f2 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:28.900792 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 28/54] 10.222.0.30 () {60 vars in 1132 bytes} [Sat Oct 15 08:31:28 2022] POST /identity/v3/users => generated 275 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:31:32.160458 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b3743-9568-4537-9981-61b161175dfa None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:32.161520 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b3743-9568-4537-9981-61b161175dfa None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:32.162038 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b3743-9568-4537-9981-61b161175dfa None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:32.163617 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 27/55] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:31:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:31:32.169566 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f204e-6caa-4665-92e1-b44dcc885ba7 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:32.170422 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f204e-6caa-4665-92e1-b44dcc885ba7 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:32.170926 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f204e-6caa-4665-92e1-b44dcc885ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:32.199307 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-8b4f204e-6caa-4665-92e1-b44dcc885ba7 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:31:32.200853 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-8b4f204e-6caa-4665-92e1-b44dcc885ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:31:32.224649 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0743320-a361-4566-838f-1ce3d992030f None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:32.225259 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0743320-a361-4566-838f-1ce3d992030f None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:32.225820 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0743320-a361-4566-838f-1ce3d992030f None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:32.227622 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 28/56] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:31:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:31:32.232355 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5672ba78-f825-46e2-9dec-98d40d60c34a None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:32.232893 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5672ba78-f825-46e2-9dec-98d40d60c34a None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:32.233373 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5672ba78-f825-46e2-9dec-98d40d60c34a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:32.252685 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-5672ba78-f825-46e2-9dec-98d40d60c34a None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:31:32.254210 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-5672ba78-f825-46e2-9dec-98d40d60c34a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:31:32.266034 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:32.310549 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 29/57] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:31:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 08:31:32.372158 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 29/58] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:31:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 08:31:32.462626 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2833bf0-538e-4f07-8500-123b07554c9a None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:32.463220 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2833bf0-538e-4f07-8500-123b07554c9a None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:32.464220 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2833bf0-538e-4f07-8500-123b07554c9a None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:32.466045 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 30/59] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:31:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:31:32.470889 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3153990-1b39-4828-8c28-5ed307754c69 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:32.471390 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3153990-1b39-4828-8c28-5ed307754c69 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:32.471885 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3153990-1b39-4828-8c28-5ed307754c69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:32.497887 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-c3153990-1b39-4828-8c28-5ed307754c69 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:31:32.499380 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-c3153990-1b39-4828-8c28-5ed307754c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:31:32.583519 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 30/60] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:31:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 08:31:32.640608 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2833bf0-538e-4f07-8500-123b07554c9a None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:32.659629 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:32.737603 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3153990-1b39-4828-8c28-5ed307754c69 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:32.779744 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2833bf0-538e-4f07-8500-123b07554c9a None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:32.820330 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b2bd7e-bdc3-49d0-a1a7-5a6d3d194a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:32.820977 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2bd7e-bdc3-49d0-a1a7-5a6d3d194a6c None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:32.821265 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2bd7e-bdc3-49d0-a1a7-5a6d3d194a6c None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:32.821543 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2bd7e-bdc3-49d0-a1a7-5a6d3d194a6c None admin] PATH_INFO: `/v3/roles/member` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:32.833519 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b2bd7e-bdc3-49d0-a1a7-5a6d3d194a6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:31:32.838852 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-09b2bd7e-bdc3-49d0-a1a7-5a6d3d194a6c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 15 08:31:32.838852 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:31:32.838852 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:31:32.838852 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:31:32.838852 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:31:32.838852 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:31:32.838852 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:31:32.838852 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:31:32.838852 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:31:32.838852 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:31:32.838852 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:31:32.838852 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:31:32.838852 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:31:32.838852 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:31:32.838852 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:31:32.838852 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:31:32.839697 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:31:32.839697 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:31:32.839697 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:31:32.839697 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:31:32.839697 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:31:32.839697 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:31:32.839697 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:31:32.839697 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:31:32.839697 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:31:32.839697 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:31:32.839697 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:31:32.839697 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:31:32.839697 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:31:32.839697 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:31:32.839697 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:31:32.839697 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:31:32.840462 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:31:32.840462 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:31:32.840462 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:31:32.840462 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:31:32.840462 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:31:32.840462 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:31:32.840462 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:31:32.840462 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:31:32.840462 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:31:32.840462 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:31:32.840462 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 15 08:31:32.840462 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:31:32.840462 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 31/61] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 08:31:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:31:32.841077 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b11d90-8e24-4f0c-8351-dd947aec3389 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:32.841347 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b11d90-8e24-4f0c-8351-dd947aec3389 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:32.841801 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b11d90-8e24-4f0c-8351-dd947aec3389 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:32.843248 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 32/62] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:31:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:31:32.844458 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0fac34-b697-4eef-9005-2bf378cb4c3e None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:32.844954 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0fac34-b697-4eef-9005-2bf378cb4c3e None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:32.845424 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0fac34-b697-4eef-9005-2bf378cb4c3e None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:32.846819 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 33/63] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:31:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:31:32.848212 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e4aa4-899b-4e76-aa22-145890478b0c None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:32.848498 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e4aa4-899b-4e76-aa22-145890478b0c None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:32.848773 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e4aa4-899b-4e76-aa22-145890478b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:32.862736 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:32.868087 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-3f5e4aa4-899b-4e76-aa22-145890478b0c None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:31:32.869288 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-3f5e4aa4-899b-4e76-aa22-145890478b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:31:32.896275 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3153990-1b39-4828-8c28-5ed307754c69 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:32.930295 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0683ab01-2273-407d-8242-8eccfbbf7ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:32.930523 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:32.930942 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0683ab01-2273-407d-8242-8eccfbbf7ada None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:32.931224 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0683ab01-2273-407d-8242-8eccfbbf7ada None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:32.931501 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0683ab01-2273-407d-8242-8eccfbbf7ada None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:32.936753 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 34/64] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:31:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 08:31:32.949828 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5e4aa4-899b-4e76-aa22-145890478b0c None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:32.949909 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0683ab01-2273-407d-8242-8eccfbbf7ada None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:31:32.949909 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-0683ab01-2273-407d-8242-8eccfbbf7ada None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 08:31:32.949909 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:31:32.949909 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:31:32.949909 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:31:32.949909 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:31:32.949909 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:31:32.949909 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:31:32.949909 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:31:32.949909 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:31:32.949909 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:31:32.949909 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:31:32.949909 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:31:32.949909 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:31:32.950624 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:31:32.950624 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:31:32.950624 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:31:32.950624 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:31:32.950624 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:31:32.950624 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:31:32.950624 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:31:32.950624 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:31:32.950624 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:31:32.950624 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:31:32.950624 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:31:32.950624 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:31:32.950624 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:31:32.950624 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:31:32.950624 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:31:32.950624 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:31:32.950624 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:31:32.951438 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:31:32.951438 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:31:32.951438 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:31:32.951438 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:31:32.951438 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:31:32.951438 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:31:32.951438 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:31:32.951438 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:31:32.951438 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:31:32.951438 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:31:32.951438 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:31:32.951438 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:31:32.951438 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 08:31:32.951438 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:31:32.951438 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 31/65] 10.222.0.30 () {56 vars in 1096 bytes} [Sat Oct 15 08:31:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 08:31:32.952147 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efa3cb7-b5b9-424f-af46-d2200366c52a None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:32.952147 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efa3cb7-b5b9-424f-af46-d2200366c52a None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:32.952147 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efa3cb7-b5b9-424f-af46-d2200366c52a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:32.968159 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-6efa3cb7-b5b9-424f-af46-d2200366c52a None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:31:32.969345 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-6efa3cb7-b5b9-424f-af46-d2200366c52a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:31:33.040988 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 32/66] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:31:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 08:31:33.042701 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5e4aa4-899b-4e76-aa22-145890478b0c None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:33.046590 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efa3cb7-b5b9-424f-af46-d2200366c52a None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:33.077974 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d55fdb-b702-4d64-b7a4-187cb529a47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:33.078900 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d55fdb-b702-4d64-b7a4-187cb529a47e None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:33.079405 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d55fdb-b702-4d64-b7a4-187cb529a47e None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:33.079870 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d55fdb-b702-4d64-b7a4-187cb529a47e None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:33.080708 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d55fdb-b702-4d64-b7a4-187cb529a47e None admin] RBAC: Adding query filter params (name=member) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:31:33.081195 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d55fdb-b702-4d64-b7a4-187cb529a47e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:33.082846 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d55fdb-b702-4d64-b7a4-187cb529a47e None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:33.095750 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 35/67] 10.222.0.30 () {56 vars in 1101 bytes} [Sat Oct 15 08:31:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:33.101337 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d55fdb-b702-4d64-b7a4-187cb529a47e None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:33.125335 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:33.173744 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efa3cb7-b5b9-424f-af46-d2200366c52a None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:33.181311 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:33.213035 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d55fdb-b702-4d64-b7a4-187cb529a47e None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:33.215658 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81046136-00f7-4501-a447-4962e4c21e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:33.216303 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81046136-00f7-4501-a447-4962e4c21e00 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:33.216589 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81046136-00f7-4501-a447-4962e4c21e00 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:33.216869 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81046136-00f7-4501-a447-4962e4c21e00 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:33.227189 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81046136-00f7-4501-a447-4962e4c21e00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:31:33.232983 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-81046136-00f7-4501-a447-4962e4c21e00 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 15 08:31:33.232983 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:31:33.232983 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:31:33.232983 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:31:33.232983 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:31:33.232983 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:31:33.232983 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:31:33.232983 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:31:33.232983 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:31:33.232983 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:31:33.232983 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:31:33.232983 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:31:33.232983 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:31:33.232983 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:31:33.232983 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:31:33.232983 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:31:33.233881 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:31:33.233881 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:31:33.233881 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:31:33.233881 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:31:33.233881 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:31:33.233881 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:31:33.233881 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:31:33.233881 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:31:33.233881 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:31:33.233881 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:31:33.233881 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:31:33.233881 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:31:33.233881 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:31:33.233881 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:31:33.233881 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:31:33.233881 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:31:33.238435 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:31:33.238435 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:31:33.238435 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:31:33.238435 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:31:33.238435 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:31:33.238435 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:31:33.238435 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:31:33.238435 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:31:33.238435 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:31:33.238435 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:31:33.238435 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 15 08:31:33.238435 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:31:33.238435 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 33/68] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 08:31:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:31:33.238435 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81046136-00f7-4501-a447-4962e4c21e00 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:33.251639 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e562440e-20d6-4327-b75d-345e7e487d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:33.252257 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e562440e-20d6-4327-b75d-345e7e487d5b None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:33.252543 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e562440e-20d6-4327-b75d-345e7e487d5b None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:33.252824 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e562440e-20d6-4327-b75d-345e7e487d5b None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:33.253491 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e562440e-20d6-4327-b75d-345e7e487d5b None admin] RBAC: Adding query filter params (name=admin) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:31:33.253834 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e562440e-20d6-4327-b75d-345e7e487d5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:33.255228 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e562440e-20d6-4327-b75d-345e7e487d5b None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:33.263515 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 36/69] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 08:31:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:33.264169 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e562440e-20d6-4327-b75d-345e7e487d5b None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:33.345898 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81046136-00f7-4501-a447-4962e4c21e00 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:33.365512 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e562440e-20d6-4327-b75d-345e7e487d5b None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:33.367949 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:33.380135 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b863076c-2516-42fb-8063-4548a88d822a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:33.380767 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b863076c-2516-42fb-8063-4548a88d822a None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:33.381049 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b863076c-2516-42fb-8063-4548a88d822a None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:33.381327 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b863076c-2516-42fb-8063-4548a88d822a None admin] PATH_INFO: `/v3/users/d4cf2957a0f5441cb522c75512551dee` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:33.390091 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:33.395047 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b863076c-2516-42fb-8063-4548a88d822a None admin] RBAC: Authorizing `identity:get_user(user_id=d4cf2957a0f5441cb522c75512551dee)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:33.396523 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b863076c-2516-42fb-8063-4548a88d822a None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:33.398229 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 34/70] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 08:31:33 2022] GET /identity/v3/users/d4cf2957a0f5441cb522c75512551dee => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:33.399499 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b863076c-2516-42fb-8063-4548a88d822a None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:33.409632 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307a32ec-f4ba-47ca-be96-cb4aef4e86c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:33.410299 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307a32ec-f4ba-47ca-be96-cb4aef4e86c7 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:33.410587 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307a32ec-f4ba-47ca-be96-cb4aef4e86c7 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:33.410871 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307a32ec-f4ba-47ca-be96-cb4aef4e86c7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:33.421551 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307a32ec-f4ba-47ca-be96-cb4aef4e86c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:31:33.426409 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-307a32ec-f4ba-47ca-be96-cb4aef4e86c7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 08:31:33.426409 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:31:33.426409 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:31:33.426409 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:31:33.426409 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:31:33.426409 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:31:33.426409 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:31:33.426409 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:31:33.426409 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:31:33.426409 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:31:33.426409 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:31:33.426409 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:31:33.426409 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:31:33.426409 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:31:33.426409 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:31:33.426409 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:31:33.427300 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:31:33.427300 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:31:33.427300 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:31:33.427300 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:31:33.427300 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:31:33.427300 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:31:33.427300 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:31:33.427300 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:31:33.427300 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:31:33.427300 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:31:33.427300 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:31:33.427300 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:31:33.427300 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:31:33.427300 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:31:33.427300 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:31:33.427300 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:31:33.428067 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:31:33.428067 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:31:33.428067 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:31:33.428067 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:31:33.428067 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:31:33.428067 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:31:33.428067 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:31:33.428067 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:31:33.428067 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:31:33.428067 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:31:33.428067 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 08:31:33.428067 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:31:33.428067 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 37/71] 10.222.0.30 () {56 vars in 1114 bytes} [Sat Oct 15 08:31:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:31:33.428710 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307a32ec-f4ba-47ca-be96-cb4aef4e86c7 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:33.502267 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b863076c-2516-42fb-8063-4548a88d822a None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:33.525439 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307a32ec-f4ba-47ca-be96-cb4aef4e86c7 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:33.537105 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd08824a-b6b5-464c-934b-f08204f39bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:33.537732 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd08824a-b6b5-464c-934b-f08204f39bc3 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:33.538021 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd08824a-b6b5-464c-934b-f08204f39bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:33.538299 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd08824a-b6b5-464c-934b-f08204f39bc3 None admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:33.538962 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd08824a-b6b5-464c-934b-f08204f39bc3 None admin] RBAC: Adding query filter params (name=member) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:31:33.539266 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd08824a-b6b5-464c-934b-f08204f39bc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:33.540902 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd08824a-b6b5-464c-934b-f08204f39bc3 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:33.549285 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 35/72] 10.222.0.30 () {56 vars in 1101 bytes} [Sat Oct 15 08:31:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:33.550100 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd08824a-b6b5-464c-934b-f08204f39bc3 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:33.561430 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e546b5b-8556-4a13-b3bb-b864e563c0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:33.562106 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e546b5b-8556-4a13-b3bb-b864e563c0c7 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:33.562398 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e546b5b-8556-4a13-b3bb-b864e563c0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:33.562685 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e546b5b-8556-4a13-b3bb-b864e563c0c7 None admin] PATH_INFO: `/v3/users/6a470f28aa544b3cb42de175f4e32d94` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:33.563833 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e546b5b-8556-4a13-b3bb-b864e563c0c7 None admin] RBAC: Authorizing `identity:get_user(user_id=6a470f28aa544b3cb42de175f4e32d94)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:33.565292 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e546b5b-8556-4a13-b3bb-b864e563c0c7 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:33.566919 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 38/73] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 08:31:33 2022] GET /identity/v3/users/6a470f28aa544b3cb42de175f4e32d94 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:33.567686 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e546b5b-8556-4a13-b3bb-b864e563c0c7 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:33.592166 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:33.617196 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:33.648739 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd08824a-b6b5-464c-934b-f08204f39bc3 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:33.667069 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e546b5b-8556-4a13-b3bb-b864e563c0c7 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:33.681538 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c481a0e-0d5f-4d11-b9ba-c8b084a7aad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:33.682203 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c481a0e-0d5f-4d11-b9ba-c8b084a7aad8 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:33.682511 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c481a0e-0d5f-4d11-b9ba-c8b084a7aad8 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:33.682792 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c481a0e-0d5f-4d11-b9ba-c8b084a7aad8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:33.683925 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c481a0e-0d5f-4d11-b9ba-c8b084a7aad8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:33.685403 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c481a0e-0d5f-4d11-b9ba-c8b084a7aad8 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:33.686997 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 36/74] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:31:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:33.691683 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c481a0e-0d5f-4d11-b9ba-c8b084a7aad8 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:33.700726 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e3f0eb-7414-45db-85bc-1922529de856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:33.701368 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e3f0eb-7414-45db-85bc-1922529de856 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:33.701658 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e3f0eb-7414-45db-85bc-1922529de856 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:33.701969 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e3f0eb-7414-45db-85bc-1922529de856 None admin] PATH_INFO: `/v3/projects/5667e30274a94890a1493a76daa4eced` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:33.703116 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e3f0eb-7414-45db-85bc-1922529de856 None admin] RBAC: Authorizing `identity:get_project(project_id=5667e30274a94890a1493a76daa4eced)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:33.704572 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e3f0eb-7414-45db-85bc-1922529de856 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:33.706238 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 39/75] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 08:31:33 2022] GET /identity/v3/projects/5667e30274a94890a1493a76daa4eced => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:33.707010 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e3f0eb-7414-45db-85bc-1922529de856 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:33.789691 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c481a0e-0d5f-4d11-b9ba-c8b084a7aad8 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:33.799599 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:33.804690 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e3f0eb-7414-45db-85bc-1922529de856 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:33.821337 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:33.823798 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3629c944-ccea-4e5f-b34a-05fff4b9dc3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:33.824410 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3629c944-ccea-4e5f-b34a-05fff4b9dc3f None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:33.824692 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3629c944-ccea-4e5f-b34a-05fff4b9dc3f None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:33.824974 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3629c944-ccea-4e5f-b34a-05fff4b9dc3f None admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:33.825632 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3629c944-ccea-4e5f-b34a-05fff4b9dc3f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:31:33.825964 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3629c944-ccea-4e5f-b34a-05fff4b9dc3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:33.827339 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3629c944-ccea-4e5f-b34a-05fff4b9dc3f None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:33.835002 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 37/76] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 08:31:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:33.835785 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3629c944-ccea-4e5f-b34a-05fff4b9dc3f None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:33.839541 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514ebd17-2f66-4c32-8953-544d90d388a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:33.840141 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514ebd17-2f66-4c32-8953-544d90d388a6 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:33.840430 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514ebd17-2f66-4c32-8953-544d90d388a6 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:33.840716 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514ebd17-2f66-4c32-8953-544d90d388a6 None admin] PATH_INFO: `/v3/users/d4cf2957a0f5441cb522c75512551dee` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:33.841886 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514ebd17-2f66-4c32-8953-544d90d388a6 None admin] RBAC: Authorizing `identity:get_user(user_id=d4cf2957a0f5441cb522c75512551dee)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:33.843374 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514ebd17-2f66-4c32-8953-544d90d388a6 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:33.849389 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 40/77] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 08:31:33 2022] GET /identity/v3/users/d4cf2957a0f5441cb522c75512551dee => generated 275 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:33.849389 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514ebd17-2f66-4c32-8953-544d90d388a6 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:33.931326 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3629c944-ccea-4e5f-b34a-05fff4b9dc3f None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:33.946567 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514ebd17-2f66-4c32-8953-544d90d388a6 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:33.963845 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd291da8-57a8-4225-82ee-d87efaf0afe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:33.964464 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd291da8-57a8-4225-82ee-d87efaf0afe7 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:33.964746 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd291da8-57a8-4225-82ee-d87efaf0afe7 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:33.965018 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd291da8-57a8-4225-82ee-d87efaf0afe7 None admin] PATH_INFO: `/v3/projects/5667e30274a94890a1493a76daa4eced` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:33.966163 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd291da8-57a8-4225-82ee-d87efaf0afe7 None admin] RBAC: Authorizing `identity:get_project(project_id=5667e30274a94890a1493a76daa4eced)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:33.967584 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd291da8-57a8-4225-82ee-d87efaf0afe7 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:33.969189 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 38/78] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 08:31:33 2022] GET /identity/v3/projects/5667e30274a94890a1493a76daa4eced => generated 298 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:33.969957 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd291da8-57a8-4225-82ee-d87efaf0afe7 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:33.984942 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb814f99-7f11-491e-b47f-c86ecbd0c8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:33.985548 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb814f99-7f11-491e-b47f-c86ecbd0c8b5 None admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:33.985859 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb814f99-7f11-491e-b47f-c86ecbd0c8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:33.986140 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb814f99-7f11-491e-b47f-c86ecbd0c8b5 None admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:33.986788 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb814f99-7f11-491e-b47f-c86ecbd0c8b5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:33.988228 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb814f99-7f11-491e-b47f-c86ecbd0c8b5 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:34.008802 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 41/79] 10.222.0.30 () {60 vars in 1132 bytes} [Sat Oct 15 08:31:33 2022] POST /identity/v3/users => generated 289 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:31:34.009609 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb814f99-7f11-491e-b47f-c86ecbd0c8b5 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:34.015945 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:34.060949 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:34.088676 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd291da8-57a8-4225-82ee-d87efaf0afe7 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:34.127573 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f1825e-6be3-4ad5-8962-e842778f1655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:34.128264 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f1825e-6be3-4ad5-8962-e842778f1655 None admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:34.128554 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f1825e-6be3-4ad5-8962-e842778f1655 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:34.128861 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f1825e-6be3-4ad5-8962-e842778f1655 None admin] PATH_INFO: `/v3/projects/5667e30274a94890a1493a76daa4eced/users/d4cf2957a0f5441cb522c75512551dee/roles/3248f0268738489b883041851d681bb7` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:34.129249 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb814f99-7f11-491e-b47f-c86ecbd0c8b5 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:34.130878 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f1825e-6be3-4ad5-8962-e842778f1655 None admin] RBAC: Authorizing `identity:create_grant(project_id=5667e30274a94890a1493a76daa4eced, user_id=d4cf2957a0f5441cb522c75512551dee, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:34.132400 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f1825e-6be3-4ad5-8962-e842778f1655 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:34.146744 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 39/80] 10.222.0.30 () {58 vars in 1439 bytes} [Sat Oct 15 08:31:33 2022] PUT /identity/v3/projects/5667e30274a94890a1493a76daa4eced/users/d4cf2957a0f5441cb522c75512551dee/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:31:34.158733 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f1825e-6be3-4ad5-8962-e842778f1655 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:34.176663 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af38f947-f11c-47da-8422-ed2d36eb976f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:34.178678 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af38f947-f11c-47da-8422-ed2d36eb976f None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:34.179210 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af38f947-f11c-47da-8422-ed2d36eb976f None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:34.179757 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af38f947-f11c-47da-8422-ed2d36eb976f None admin] PATH_INFO: `/v3/users/d4cf2957a0f5441cb522c75512551dee` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:34.181506 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af38f947-f11c-47da-8422-ed2d36eb976f None admin] RBAC: Authorizing `identity:get_user(user_id=d4cf2957a0f5441cb522c75512551dee)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:34.184525 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af38f947-f11c-47da-8422-ed2d36eb976f None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:34.186358 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 42/81] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 08:31:34 2022] GET /identity/v3/users/d4cf2957a0f5441cb522c75512551dee => generated 275 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:34.187167 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af38f947-f11c-47da-8422-ed2d36eb976f None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:34.324430 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:34.328837 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f1825e-6be3-4ad5-8962-e842778f1655 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:34.339646 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af38f947-f11c-47da-8422-ed2d36eb976f None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:34.355780 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:34.377136 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc37faa1-79e3-4c3c-9906-f1411a619109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:34.379563 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc37faa1-79e3-4c3c-9906-f1411a619109 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:34.379563 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc37faa1-79e3-4c3c-9906-f1411a619109 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:34.379563 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc37faa1-79e3-4c3c-9906-f1411a619109 None admin] PATH_INFO: `/v3/projects/167e11807cf7410b9f0eb9c73d18c927` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:34.379830 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc37faa1-79e3-4c3c-9906-f1411a619109 None admin] RBAC: Authorizing `identity:get_project(project_id=167e11807cf7410b9f0eb9c73d18c927)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:34.381460 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc37faa1-79e3-4c3c-9906-f1411a619109 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:34.383662 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 40/82] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 08:31:34 2022] GET /identity/v3/projects/167e11807cf7410b9f0eb9c73d18c927 => generated 312 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:34.384622 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc37faa1-79e3-4c3c-9906-f1411a619109 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:34.393272 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf51bb2-fdf3-4d5e-ae0b-8e004d15d8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:34.394743 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf51bb2-fdf3-4d5e-ae0b-8e004d15d8f3 None admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:34.395197 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf51bb2-fdf3-4d5e-ae0b-8e004d15d8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:34.395633 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf51bb2-fdf3-4d5e-ae0b-8e004d15d8f3 None admin] PATH_INFO: `/v3/projects/5667e30274a94890a1493a76daa4eced/users/6a470f28aa544b3cb42de175f4e32d94/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:34.400412 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf51bb2-fdf3-4d5e-ae0b-8e004d15d8f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=5667e30274a94890a1493a76daa4eced, user_id=6a470f28aa544b3cb42de175f4e32d94, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:34.402904 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf51bb2-fdf3-4d5e-ae0b-8e004d15d8f3 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:34.426953 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 43/83] 10.222.0.30 () {58 vars in 1439 bytes} [Sat Oct 15 08:31:34 2022] PUT /identity/v3/projects/5667e30274a94890a1493a76daa4eced/users/6a470f28aa544b3cb42de175f4e32d94/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:31:34.428307 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf51bb2-fdf3-4d5e-ae0b-8e004d15d8f3 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:34.597594 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf51bb2-fdf3-4d5e-ae0b-8e004d15d8f3 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:34.620367 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc37faa1-79e3-4c3c-9906-f1411a619109 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:34.680805 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc117c64-6f3d-4fc0-a721-7305603ffbe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:34.681459 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc117c64-6f3d-4fc0-a721-7305603ffbe9 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:34.681771 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc117c64-6f3d-4fc0-a721-7305603ffbe9 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:34.682053 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc117c64-6f3d-4fc0-a721-7305603ffbe9 None admin] PATH_INFO: `/v3/projects/5667e30274a94890a1493a76daa4eced` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:34.683356 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc117c64-6f3d-4fc0-a721-7305603ffbe9 None admin] RBAC: Authorizing `identity:get_project(project_id=5667e30274a94890a1493a76daa4eced)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:34.684843 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc117c64-6f3d-4fc0-a721-7305603ffbe9 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:34.689201 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 41/84] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 08:31:34 2022] GET /identity/v3/projects/5667e30274a94890a1493a76daa4eced => generated 298 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:34.694685 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0c3ce7-d6f0-48a6-9ad3-7407f12ca34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:34.694833 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc117c64-6f3d-4fc0-a721-7305603ffbe9 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:34.695396 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0c3ce7-d6f0-48a6-9ad3-7407f12ca34c None admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:34.695698 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0c3ce7-d6f0-48a6-9ad3-7407f12ca34c None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:34.695973 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0c3ce7-d6f0-48a6-9ad3-7407f12ca34c None admin] PATH_INFO: `/v3/projects/167e11807cf7410b9f0eb9c73d18c927/users/d4cf2957a0f5441cb522c75512551dee/roles/3248f0268738489b883041851d681bb7` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:34.698535 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0c3ce7-d6f0-48a6-9ad3-7407f12ca34c None admin] RBAC: Authorizing `identity:create_grant(project_id=167e11807cf7410b9f0eb9c73d18c927, user_id=d4cf2957a0f5441cb522c75512551dee, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:34.700066 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0c3ce7-d6f0-48a6-9ad3-7407f12ca34c None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:34.722348 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 44/85] 10.222.0.30 () {58 vars in 1439 bytes} [Sat Oct 15 08:31:34 2022] PUT /identity/v3/projects/167e11807cf7410b9f0eb9c73d18c927/users/d4cf2957a0f5441cb522c75512551dee/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:31:34.763839 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:34.849463 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc117c64-6f3d-4fc0-a721-7305603ffbe9 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:34.887056 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f761e78-a97a-4ed4-a2da-d403d0d9b90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:34.887742 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f761e78-a97a-4ed4-a2da-d403d0d9b90f None admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:34.888030 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f761e78-a97a-4ed4-a2da-d403d0d9b90f None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:34.888312 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f761e78-a97a-4ed4-a2da-d403d0d9b90f None admin] PATH_INFO: `/v3/projects/5667e30274a94890a1493a76daa4eced/users/d4cf2957a0f5441cb522c75512551dee/roles/d236b691c1434103b73cc4fd0b39fec1` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:34.890546 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f761e78-a97a-4ed4-a2da-d403d0d9b90f None admin] RBAC: Authorizing `identity:create_grant(project_id=5667e30274a94890a1493a76daa4eced, user_id=d4cf2957a0f5441cb522c75512551dee, role_id=d236b691c1434103b73cc4fd0b39fec1)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:34.892263 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f761e78-a97a-4ed4-a2da-d403d0d9b90f None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:34.914000 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 42/86] 10.222.0.30 () {58 vars in 1439 bytes} [Sat Oct 15 08:31:34 2022] PUT /identity/v3/projects/5667e30274a94890a1493a76daa4eced/users/d4cf2957a0f5441cb522c75512551dee/roles/d236b691c1434103b73cc4fd0b39fec1 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 08:31:36.911923 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6dccf2-5747-485a-bcaa-ca5cc9f1d9e0 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:36.912439 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6dccf2-5747-485a-bcaa-ca5cc9f1d9e0 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:36.912512 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6dccf2-5747-485a-bcaa-ca5cc9f1d9e0 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:36.913832 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 45/87] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:31:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:31:36.927293 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32acd485-1be6-4e1a-a7cc-5118f6281e08 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:36.928134 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32acd485-1be6-4e1a-a7cc-5118f6281e08 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:36.928610 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32acd485-1be6-4e1a-a7cc-5118f6281e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:36.949393 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-32acd485-1be6-4e1a-a7cc-5118f6281e08 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:31:36.950878 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-32acd485-1be6-4e1a-a7cc-5118f6281e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:31:37.039969 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 43/88] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:31:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 08:31:37.418364 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087e9fc-3fbd-4e7c-95d7-64403e95acd7 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:37.418988 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087e9fc-3fbd-4e7c-95d7-64403e95acd7 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:37.419461 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087e9fc-3fbd-4e7c-95d7-64403e95acd7 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:37.421030 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 46/89] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:31:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:31:37.426394 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae6ae3e-e4bb-4002-a162-2a2bc5042781 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:37.426691 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae6ae3e-e4bb-4002-a162-2a2bc5042781 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:37.426970 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae6ae3e-e4bb-4002-a162-2a2bc5042781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:37.450473 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-dae6ae3e-e4bb-4002-a162-2a2bc5042781 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:31:37.451669 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-dae6ae3e-e4bb-4002-a162-2a2bc5042781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:31:37.479987 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:37.485554 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb5291-b01b-4ce0-b06a-6ae83246d2b1 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:37.486098 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb5291-b01b-4ce0-b06a-6ae83246d2b1 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:37.486958 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb5291-b01b-4ce0-b06a-6ae83246d2b1 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:37.488790 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 47/90] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:31:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 08:31:37.495330 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaa59a8-9cef-4c44-86ca-4f776f098f57 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:37.495853 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaa59a8-9cef-4c44-86ca-4f776f098f57 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:37.496323 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaa59a8-9cef-4c44-86ca-4f776f098f57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:37.530729 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-5aaa59a8-9cef-4c44-86ca-4f776f098f57 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:31:37.532628 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-5aaa59a8-9cef-4c44-86ca-4f776f098f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:31:37.559416 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 44/91] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:31:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 08:31:37.560323 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae6ae3e-e4bb-4002-a162-2a2bc5042781 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:37.560665 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:37.650108 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 48/92] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:31:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 08:31:37.729794 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0b71d4-7a58-414f-947a-25119fb69748 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:37.730453 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0b71d4-7a58-414f-947a-25119fb69748 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:37.731183 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0b71d4-7a58-414f-947a-25119fb69748 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:37.733349 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 49/93] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:31:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:31:37.739429 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0a0de8-d149-40e4-9f0c-d97a55b8e213 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:37.739976 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0a0de8-d149-40e4-9f0c-d97a55b8e213 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:37.740511 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0a0de8-d149-40e4-9f0c-d97a55b8e213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:37.762889 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae6ae3e-e4bb-4002-a162-2a2bc5042781 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:37.775628 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-df0a0de8-d149-40e4-9f0c-d97a55b8e213 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:31:37.777575 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-df0a0de8-d149-40e4-9f0c-d97a55b8e213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:31:37.825709 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079718db-697d-45a9-92e3-879e78d81a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:37.826362 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079718db-697d-45a9-92e3-879e78d81a6d None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:37.826650 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079718db-697d-45a9-92e3-879e78d81a6d None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:37.826929 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079718db-697d-45a9-92e3-879e78d81a6d None admin] PATH_INFO: `/v3/roles/member` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:37.844537 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079718db-697d-45a9-92e3-879e78d81a6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:31:37.849494 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-079718db-697d-45a9-92e3-879e78d81a6d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 15 08:31:37.849494 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:31:37.849494 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:31:37.849494 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:31:37.849494 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:31:37.849494 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:31:37.849494 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:31:37.849494 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:31:37.849494 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:31:37.849494 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:31:37.849494 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:31:37.849494 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:31:37.849494 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:31:37.849494 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:31:37.849494 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:31:37.849494 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:31:37.850348 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:31:37.850348 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:31:37.850348 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:31:37.850348 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:31:37.850348 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:31:37.850348 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:31:37.850348 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:31:37.850348 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:31:37.850348 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:31:37.850348 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:31:37.850348 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:31:37.850348 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:31:37.850348 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:31:37.850348 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:31:37.850348 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:31:37.850348 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:31:37.851109 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:31:37.851109 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:31:37.851109 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:31:37.851109 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:31:37.851109 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:31:37.851109 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:31:37.851109 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:31:37.851109 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:31:37.851109 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:31:37.851109 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:31:37.851109 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 15 08:31:37.851109 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:31:37.857792 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 45/94] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 08:31:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:31:37.860353 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 50/95] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:31:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 08:31:37.863570 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079718db-697d-45a9-92e3-879e78d81a6d None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:37.884556 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:38.002464 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079718db-697d-45a9-92e3-879e78d81a6d None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:38.049872 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb8c252-f130-4ea2-a8a0-49a39bbbd7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:38.050496 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb8c252-f130-4ea2-a8a0-49a39bbbd7e4 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:38.050786 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb8c252-f130-4ea2-a8a0-49a39bbbd7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:38.051068 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb8c252-f130-4ea2-a8a0-49a39bbbd7e4 None admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:38.051741 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb8c252-f130-4ea2-a8a0-49a39bbbd7e4 None admin] RBAC: Adding query filter params (name=member) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:31:38.052053 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb8c252-f130-4ea2-a8a0-49a39bbbd7e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:38.053436 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb8c252-f130-4ea2-a8a0-49a39bbbd7e4 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:38.059164 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0a0de8-d149-40e4-9f0c-d97a55b8e213 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:38.064043 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 46/96] 10.222.0.30 () {56 vars in 1101 bytes} [Sat Oct 15 08:31:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:38.065280 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ce85b-3cc9-44fd-9a13-ca60e90ab2ce None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:38.065571 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ce85b-3cc9-44fd-9a13-ca60e90ab2ce None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:38.065886 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ce85b-3cc9-44fd-9a13-ca60e90ab2ce None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:38.067167 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 47/97] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:31:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:31:38.067869 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4ce85b-3cc9-44fd-9a13-ca60e90ab2ce None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:38.168347 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:38.173228 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:38.186070 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0a0de8-d149-40e4-9f0c-d97a55b8e213 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:38.197228 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4ce85b-3cc9-44fd-9a13-ca60e90ab2ce None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:38.225207 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30132a74-7620-4011-898b-1d9cb1aeb66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:38.226271 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30132a74-7620-4011-898b-1d9cb1aeb66e None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:38.226772 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30132a74-7620-4011-898b-1d9cb1aeb66e None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:38.227220 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30132a74-7620-4011-898b-1d9cb1aeb66e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:38.239170 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30132a74-7620-4011-898b-1d9cb1aeb66e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:31:38.239649 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d51267b-e760-43ef-9185-57099dafb8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:38.240291 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d51267b-e760-43ef-9185-57099dafb8fd None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:38.240575 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d51267b-e760-43ef-9185-57099dafb8fd None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:38.240852 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d51267b-e760-43ef-9185-57099dafb8fd None admin] PATH_INFO: `/v3/domains/default` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:38.242263 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d51267b-e760-43ef-9185-57099dafb8fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:38.243712 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d51267b-e760-43ef-9185-57099dafb8fd None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:38.245872 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 48/98] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:31:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:38.246468 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-30132a74-7620-4011-898b-1d9cb1aeb66e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 08:31:38.246468 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:31:38.246468 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:31:38.246468 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:31:38.246468 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:31:38.246468 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:31:38.246468 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:31:38.246468 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:31:38.246468 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:31:38.246468 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:31:38.246468 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:31:38.246468 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:31:38.246468 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:31:38.246468 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:31:38.246468 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:31:38.246468 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:31:38.247519 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:31:38.247519 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:31:38.247519 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:31:38.247519 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:31:38.247519 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:31:38.247519 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:31:38.247519 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:31:38.247519 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:31:38.247519 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:31:38.247519 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:31:38.247519 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:31:38.247519 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:31:38.247519 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:31:38.247519 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:31:38.247519 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:31:38.247519 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:31:38.248441 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:31:38.248441 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:31:38.248441 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:31:38.248441 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:31:38.248441 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:31:38.248441 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:31:38.248441 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:31:38.248441 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:31:38.248441 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:31:38.248441 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:31:38.248441 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 08:31:38.248441 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:31:38.250265 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 51/99] 10.222.0.30 () {56 vars in 1096 bytes} [Sat Oct 15 08:31:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 08:31:38.255391 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef1f880-ecf1-4566-a046-75da5e1411ee None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:38.255391 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef1f880-ecf1-4566-a046-75da5e1411ee None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:38.255391 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef1f880-ecf1-4566-a046-75da5e1411ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:38.256562 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30132a74-7620-4011-898b-1d9cb1aeb66e None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:38.272648 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-5ef1f880-ecf1-4566-a046-75da5e1411ee None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:31:38.274071 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-5ef1f880-ecf1-4566-a046-75da5e1411ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:31:38.361494 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 49/100] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:31:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 08:31:38.361494 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef1f880-ecf1-4566-a046-75da5e1411ee None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:38.378302 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30132a74-7620-4011-898b-1d9cb1aeb66e None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:38.421387 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb158d33-ceec-48dc-bfe2-997fe356148b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:38.422060 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb158d33-ceec-48dc-bfe2-997fe356148b None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:38.422409 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb158d33-ceec-48dc-bfe2-997fe356148b None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:38.422976 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb158d33-ceec-48dc-bfe2-997fe356148b None admin] PATH_INFO: `/v3/users/d4cf2957a0f5441cb522c75512551dee` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:38.424543 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb158d33-ceec-48dc-bfe2-997fe356148b None admin] RBAC: Authorizing `identity:get_user(user_id=d4cf2957a0f5441cb522c75512551dee)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:38.426146 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb158d33-ceec-48dc-bfe2-997fe356148b None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:38.428529 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 52/101] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 08:31:38 2022] GET /identity/v3/users/d4cf2957a0f5441cb522c75512551dee => generated 275 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:38.433508 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb158d33-ceec-48dc-bfe2-997fe356148b None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:38.453623 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:38.453907 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:38.478230 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef1f880-ecf1-4566-a046-75da5e1411ee None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:38.513723 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf499e13-dc77-4849-a0fe-0fd6a7fd61ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:38.514281 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf499e13-dc77-4849-a0fe-0fd6a7fd61ec None admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:38.514561 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf499e13-dc77-4849-a0fe-0fd6a7fd61ec None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:38.514832 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf499e13-dc77-4849-a0fe-0fd6a7fd61ec None admin] PATH_INFO: `/v3/projects` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:38.515486 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf499e13-dc77-4849-a0fe-0fd6a7fd61ec None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:38.517172 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf499e13-dc77-4849-a0fe-0fd6a7fd61ec None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:38.546665 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb158d33-ceec-48dc-bfe2-997fe356148b None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:38.549842 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 50/102] 10.222.0.30 () {60 vars in 1141 bytes} [Sat Oct 15 08:31:38 2022] POST /identity/v3/projects => generated 302 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:31:38.550966 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf499e13-dc77-4849-a0fe-0fd6a7fd61ec None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:38.595622 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dd17e9-1cc1-47e4-9005-b20ea4d990f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:38.596750 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dd17e9-1cc1-47e4-9005-b20ea4d990f1 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:38.597344 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dd17e9-1cc1-47e4-9005-b20ea4d990f1 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:38.597956 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dd17e9-1cc1-47e4-9005-b20ea4d990f1 None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:38.598921 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dd17e9-1cc1-47e4-9005-b20ea4d990f1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:31:38.599549 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dd17e9-1cc1-47e4-9005-b20ea4d990f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:38.601180 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dd17e9-1cc1-47e4-9005-b20ea4d990f1 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:38.612180 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 53/103] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 08:31:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:38.614841 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3167d297-439c-4ac0-ab48-3265883168f0 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:38.615683 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3167d297-439c-4ac0-ab48-3265883168f0 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:38.616168 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3167d297-439c-4ac0-ab48-3265883168f0 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:38.618419 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 54/104] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:31:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:31:38.619550 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3167d297-439c-4ac0-ab48-3265883168f0 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:38.669205 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf499e13-dc77-4849-a0fe-0fd6a7fd61ec None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:38.699590 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:38.709560 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978bb8c6-a92c-4e55-ba77-36449c159f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:38.710260 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978bb8c6-a92c-4e55-ba77-36449c159f7b None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:38.710547 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978bb8c6-a92c-4e55-ba77-36449c159f7b None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:38.710833 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978bb8c6-a92c-4e55-ba77-36449c159f7b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:38.722501 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978bb8c6-a92c-4e55-ba77-36449c159f7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:31:38.726137 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:38.728035 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-978bb8c6-a92c-4e55-ba77-36449c159f7b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 15 08:31:38.728035 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:31:38.728035 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:31:38.728035 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:31:38.728035 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:31:38.728035 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:31:38.728035 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:31:38.728035 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:31:38.728035 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:31:38.728035 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:31:38.728035 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:31:38.728035 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:31:38.728035 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:31:38.728035 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:31:38.728035 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:31:38.728035 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:31:38.731126 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:31:38.731126 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:31:38.731126 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:31:38.731126 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:31:38.731126 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:31:38.731126 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:31:38.731126 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:31:38.731126 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:31:38.731126 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:31:38.731126 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:31:38.731126 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:31:38.731126 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:31:38.731126 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:31:38.731126 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:31:38.731126 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:31:38.731126 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:31:38.731881 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:31:38.731881 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:31:38.731881 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:31:38.731881 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:31:38.731881 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:31:38.731881 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:31:38.731881 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:31:38.731881 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:31:38.731881 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:31:38.731881 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:31:38.731881 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 15 08:31:38.731881 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:31:38.731881 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 51/105] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 08:31:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:31:38.731881 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978bb8c6-a92c-4e55-ba77-36449c159f7b None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:38.754394 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3167d297-439c-4ac0-ab48-3265883168f0 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:38.797088 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffe08cb-2470-4dac-9261-3facb9cb2db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:38.798150 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe08cb-2470-4dac-9261-3facb9cb2db0 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:38.798446 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe08cb-2470-4dac-9261-3facb9cb2db0 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:38.798725 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe08cb-2470-4dac-9261-3facb9cb2db0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:38.844681 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978bb8c6-a92c-4e55-ba77-36449c159f7b None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:38.873991 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffe08cb-2470-4dac-9261-3facb9cb2db0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:38.875452 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffe08cb-2470-4dac-9261-3facb9cb2db0 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:38.878905 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029857fa-91db-4b11-a29a-51e0e9aea1a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:38.879544 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029857fa-91db-4b11-a29a-51e0e9aea1a2 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:38.879832 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029857fa-91db-4b11-a29a-51e0e9aea1a2 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:38.880110 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029857fa-91db-4b11-a29a-51e0e9aea1a2 None admin] PATH_INFO: `/v3/users/6a470f28aa544b3cb42de175f4e32d94` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:38.881243 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029857fa-91db-4b11-a29a-51e0e9aea1a2 None admin] RBAC: Authorizing `identity:get_user(user_id=6a470f28aa544b3cb42de175f4e32d94)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:38.882732 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029857fa-91db-4b11-a29a-51e0e9aea1a2 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:38.889450 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 52/106] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 08:31:38 2022] GET /identity/v3/users/6a470f28aa544b3cb42de175f4e32d94 => generated 247 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:38.889450 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70af2e5-8b7f-46cd-b301-57da209a9f05 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:38.889450 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70af2e5-8b7f-46cd-b301-57da209a9f05 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:38.889450 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70af2e5-8b7f-46cd-b301-57da209a9f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:38.904829 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-d70af2e5-8b7f-46cd-b301-57da209a9f05 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:31:38.906048 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-d70af2e5-8b7f-46cd-b301-57da209a9f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:31:38.959305 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:38.964687 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:38.979736 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 53/107] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:31:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 08:31:38.980597 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70af2e5-8b7f-46cd-b301-57da209a9f05 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:38.994425 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 55/108] 10.222.0.30 () {58 vars in 1282 bytes} [Sat Oct 15 08:31:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 08:31:38.996039 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffe08cb-2470-4dac-9261-3facb9cb2db0 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:39.099909 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70af2e5-8b7f-46cd-b301-57da209a9f05 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:39.118640 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffe08cb-2470-4dac-9261-3facb9cb2db0 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:39.147736 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6651428-a518-4142-9177-26f36f075e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:39.148646 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6651428-a518-4142-9177-26f36f075e96 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:39.148968 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6651428-a518-4142-9177-26f36f075e96 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:39.149252 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6651428-a518-4142-9177-26f36f075e96 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:39.156094 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8e4aa7-54ac-42bd-af02-6a7d6aeb01d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:39.157073 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8e4aa7-54ac-42bd-af02-6a7d6aeb01d0 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:39.157567 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8e4aa7-54ac-42bd-af02-6a7d6aeb01d0 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:39.158063 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8e4aa7-54ac-42bd-af02-6a7d6aeb01d0 None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:39.158913 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8e4aa7-54ac-42bd-af02-6a7d6aeb01d0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:31:39.159421 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8e4aa7-54ac-42bd-af02-6a7d6aeb01d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:39.160983 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8e4aa7-54ac-42bd-af02-6a7d6aeb01d0 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:39.164003 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6651428-a518-4142-9177-26f36f075e96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:31:39.169882 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-d6651428-a518-4142-9177-26f36f075e96 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 08:31:39.169882 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:31:39.169882 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:31:39.169882 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:31:39.169882 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:31:39.169882 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:31:39.169882 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:31:39.169882 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:31:39.169882 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:31:39.169882 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:31:39.169882 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:31:39.169882 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:31:39.169882 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:31:39.169882 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:31:39.169882 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:31:39.169882 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:31:39.174581 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:31:39.174581 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:31:39.174581 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:31:39.174581 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:31:39.174581 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:31:39.174581 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:31:39.174581 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:31:39.174581 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:31:39.174581 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:31:39.174581 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:31:39.174581 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:31:39.174581 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:31:39.174581 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:31:39.174581 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:31:39.174581 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:31:39.174581 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:31:39.175331 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:31:39.175331 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:31:39.175331 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:31:39.175331 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:31:39.175331 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:31:39.175331 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:31:39.175331 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:31:39.175331 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:31:39.175331 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:31:39.175331 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:31:39.175331 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 08:31:39.175331 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:31:39.175919 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 54/109] 10.222.0.30 () {56 vars in 1114 bytes} [Sat Oct 15 08:31:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:31:39.176547 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 56/110] 10.222.0.30 () {56 vars in 1101 bytes} [Sat Oct 15 08:31:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:39.176842 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6651428-a518-4142-9177-26f36f075e96 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:39.178117 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8e4aa7-54ac-42bd-af02-6a7d6aeb01d0 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:39.225752 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:39.226607 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:39.284532 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6651428-a518-4142-9177-26f36f075e96 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:39.287470 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8e4aa7-54ac-42bd-af02-6a7d6aeb01d0 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:39.321666 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75717f69-f527-4550-9713-10063f07ed82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:39.322411 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75717f69-f527-4550-9713-10063f07ed82 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:39.322814 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75717f69-f527-4550-9713-10063f07ed82 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:39.323324 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75717f69-f527-4550-9713-10063f07ed82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:39.323745 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ca9832-2b61-410b-8050-edee13ef7d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:39.324451 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ca9832-2b61-410b-8050-edee13ef7d66 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:39.324808 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ca9832-2b61-410b-8050-edee13ef7d66 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:39.325156 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ca9832-2b61-410b-8050-edee13ef7d66 None admin] PATH_INFO: `/v3/projects/5667e30274a94890a1493a76daa4eced` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:39.326385 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ca9832-2b61-410b-8050-edee13ef7d66 None admin] RBAC: Authorizing `identity:get_project(project_id=5667e30274a94890a1493a76daa4eced)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:39.327911 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ca9832-2b61-410b-8050-edee13ef7d66 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:39.333974 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 57/111] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 08:31:39 2022] GET /identity/v3/projects/5667e30274a94890a1493a76daa4eced => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:39.333974 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ca9832-2b61-410b-8050-edee13ef7d66 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:39.397937 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75717f69-f527-4550-9713-10063f07ed82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:39.399432 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75717f69-f527-4550-9713-10063f07ed82 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:39.445007 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ca9832-2b61-410b-8050-edee13ef7d66 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:39.447415 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:39.455889 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:39.480683 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880c40b7-b06d-4377-bcc1-565fc47eac7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:39.481424 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880c40b7-b06d-4377-bcc1-565fc47eac7e None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:39.481806 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880c40b7-b06d-4377-bcc1-565fc47eac7e None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:39.482174 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880c40b7-b06d-4377-bcc1-565fc47eac7e None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:39.482865 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880c40b7-b06d-4377-bcc1-565fc47eac7e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:31:39.483174 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880c40b7-b06d-4377-bcc1-565fc47eac7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:39.484602 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880c40b7-b06d-4377-bcc1-565fc47eac7e None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:39.493716 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 58/112] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 08:31:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:39.494471 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880c40b7-b06d-4377-bcc1-565fc47eac7e None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:39.517809 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 55/113] 10.222.0.30 () {58 vars in 1282 bytes} [Sat Oct 15 08:31:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 08:31:39.517809 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75717f69-f527-4550-9713-10063f07ed82 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:39.607043 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880c40b7-b06d-4377-bcc1-565fc47eac7e None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:39.622493 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75717f69-f527-4550-9713-10063f07ed82 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:39.643324 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ff69ab-657b-4de0-8be9-8bbbc4bd54dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:39.644096 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ff69ab-657b-4de0-8be9-8bbbc4bd54dc None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:39.644389 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ff69ab-657b-4de0-8be9-8bbbc4bd54dc None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:39.644664 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ff69ab-657b-4de0-8be9-8bbbc4bd54dc None admin] PATH_INFO: `/v3/users/eeb31007ce8f47d69b196d7bd51f82a7` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:39.656407 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56e432d-60a7-4897-a09b-b9fba7ef2cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:39.657108 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56e432d-60a7-4897-a09b-b9fba7ef2cbf None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:39.657462 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56e432d-60a7-4897-a09b-b9fba7ef2cbf None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:39.657838 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56e432d-60a7-4897-a09b-b9fba7ef2cbf None admin] PATH_INFO: `/v3/roles/member` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:39.659941 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ff69ab-657b-4de0-8be9-8bbbc4bd54dc None admin] RBAC: Authorizing `identity:get_user(user_id=eeb31007ce8f47d69b196d7bd51f82a7)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:39.661381 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ff69ab-657b-4de0-8be9-8bbbc4bd54dc None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:39.663013 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 59/114] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 08:31:39 2022] GET /identity/v3/users/eeb31007ce8f47d69b196d7bd51f82a7 => generated 289 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:39.663788 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ff69ab-657b-4de0-8be9-8bbbc4bd54dc None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:39.673741 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56e432d-60a7-4897-a09b-b9fba7ef2cbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:31:39.678594 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-c56e432d-60a7-4897-a09b-b9fba7ef2cbf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 15 08:31:39.678594 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:31:39.678594 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:31:39.678594 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:31:39.678594 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:31:39.678594 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:31:39.678594 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:31:39.678594 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:31:39.678594 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:31:39.678594 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:31:39.678594 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:31:39.678594 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:31:39.678594 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:31:39.678594 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:31:39.678594 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:31:39.678594 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:31:39.679657 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:31:39.679657 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:31:39.679657 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:31:39.679657 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:31:39.679657 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:31:39.679657 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:31:39.679657 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:31:39.679657 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:31:39.679657 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:31:39.679657 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:31:39.679657 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:31:39.679657 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:31:39.679657 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:31:39.679657 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:31:39.679657 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:31:39.679657 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:31:39.680599 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:31:39.680599 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:31:39.680599 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:31:39.680599 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:31:39.680599 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:31:39.680599 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:31:39.680599 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:31:39.680599 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:31:39.680599 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:31:39.680599 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:31:39.680599 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 15 08:31:39.680599 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:31:39.684983 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 56/115] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 08:31:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:31:39.684983 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56e432d-60a7-4897-a09b-b9fba7ef2cbf None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:39.691803 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:39.701143 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:39.778389 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ff69ab-657b-4de0-8be9-8bbbc4bd54dc None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:39.798722 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56e432d-60a7-4897-a09b-b9fba7ef2cbf None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:39.816336 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d45c628-2f58-4435-b656-20c49c398f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:39.817087 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d45c628-2f58-4435-b656-20c49c398f79 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:39.817465 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d45c628-2f58-4435-b656-20c49c398f79 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:39.817872 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d45c628-2f58-4435-b656-20c49c398f79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:39.818696 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d45c628-2f58-4435-b656-20c49c398f79 None admin] RBAC: Adding query filter params (role.id=3248f0268738489b883041851d681bb7, scope.project.id=5667e30274a94890a1493a76daa4eced, user.id=d4cf2957a0f5441cb522c75512551dee) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:31:39.819107 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d45c628-2f58-4435-b656-20c49c398f79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:39.820636 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d45c628-2f58-4435-b656-20c49c398f79 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:39.827387 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 60/116] 10.222.0.30 () {56 vars in 1374 bytes} [Sat Oct 15 08:31:39 2022] GET /identity/v3/role_assignments?user.id=d4cf2957a0f5441cb522c75512551dee&scope.project.id=5667e30274a94890a1493a76daa4eced&role.id=3248f0268738489b883041851d681bb7 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:39.828124 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d45c628-2f58-4435-b656-20c49c398f79 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:39.843834 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5c36db-3e1b-49f3-b91d-0d7407c9757e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:39.845074 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5c36db-3e1b-49f3-b91d-0d7407c9757e None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:39.846120 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5c36db-3e1b-49f3-b91d-0d7407c9757e None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:39.846563 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5c36db-3e1b-49f3-b91d-0d7407c9757e None admin] PATH_INFO: `/v3/users/d4cf2957a0f5441cb522c75512551dee` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:39.848400 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5c36db-3e1b-49f3-b91d-0d7407c9757e None admin] RBAC: Authorizing `identity:get_user(user_id=d4cf2957a0f5441cb522c75512551dee)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:39.850882 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5c36db-3e1b-49f3-b91d-0d7407c9757e None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:39.852818 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 57/117] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 08:31:39 2022] GET /identity/v3/users/d4cf2957a0f5441cb522c75512551dee => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:39.854569 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5c36db-3e1b-49f3-b91d-0d7407c9757e None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:39.946691 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:39.958019 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:39.966790 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d45c628-2f58-4435-b656-20c49c398f79 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:39.988357 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5c36db-3e1b-49f3-b91d-0d7407c9757e None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:40.015827 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c87af1-1129-4f3a-9c2c-915e0cbf41d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:40.018208 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c87af1-1129-4f3a-9c2c-915e0cbf41d0 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:40.018502 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c87af1-1129-4f3a-9c2c-915e0cbf41d0 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:40.018776 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c87af1-1129-4f3a-9c2c-915e0cbf41d0 None admin] PATH_INFO: `/v3/projects/5667e30274a94890a1493a76daa4eced` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:40.023612 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c87af1-1129-4f3a-9c2c-915e0cbf41d0 None admin] RBAC: Authorizing `identity:get_project(project_id=5667e30274a94890a1493a76daa4eced)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:40.023612 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c87af1-1129-4f3a-9c2c-915e0cbf41d0 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:40.023900 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 61/118] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 08:31:39 2022] GET /identity/v3/projects/5667e30274a94890a1493a76daa4eced => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:40.024684 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c87af1-1129-4f3a-9c2c-915e0cbf41d0 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:40.038045 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0d037b-073a-426f-af0e-9052cd89bc06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:40.039121 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d037b-073a-426f-af0e-9052cd89bc06 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:40.039416 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d037b-073a-426f-af0e-9052cd89bc06 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:40.039725 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d037b-073a-426f-af0e-9052cd89bc06 None admin] PATH_INFO: `/v3/projects/5667e30274a94890a1493a76daa4eced` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:40.043221 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0d037b-073a-426f-af0e-9052cd89bc06 None admin] RBAC: Authorizing `identity:get_project(project_id=5667e30274a94890a1493a76daa4eced)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:40.044437 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0d037b-073a-426f-af0e-9052cd89bc06 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:40.046422 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 58/119] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 08:31:39 2022] GET /identity/v3/projects/5667e30274a94890a1493a76daa4eced => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:40.047302 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0d037b-073a-426f-af0e-9052cd89bc06 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:40.155028 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c87af1-1129-4f3a-9c2c-915e0cbf41d0 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:40.161735 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0d037b-073a-426f-af0e-9052cd89bc06 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:40.190417 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7560bcb4-d250-442f-b0e6-ade1abbf736f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:40.191027 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7560bcb4-d250-442f-b0e6-ade1abbf736f None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:40.191311 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7560bcb4-d250-442f-b0e6-ade1abbf736f None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:40.191585 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7560bcb4-d250-442f-b0e6-ade1abbf736f None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:40.192439 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7560bcb4-d250-442f-b0e6-ade1abbf736f None admin] RBAC: Adding query filter params (name=member) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:31:40.192753 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7560bcb4-d250-442f-b0e6-ade1abbf736f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:40.194343 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7560bcb4-d250-442f-b0e6-ade1abbf736f None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:40.194744 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efa32c6-9b36-43f9-bf9f-f3b1fb94a81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:40.195355 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa32c6-9b36-43f9-bf9f-f3b1fb94a81c None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:40.195650 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa32c6-9b36-43f9-bf9f-f3b1fb94a81c None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:40.195935 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa32c6-9b36-43f9-bf9f-f3b1fb94a81c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:40.203856 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 62/120] 10.222.0.30 () {56 vars in 1101 bytes} [Sat Oct 15 08:31:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:40.205014 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7560bcb4-d250-442f-b0e6-ade1abbf736f None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:40.212954 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:40.234258 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:40.271318 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efa32c6-9b36-43f9-bf9f-f3b1fb94a81c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:40.272784 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efa32c6-9b36-43f9-bf9f-f3b1fb94a81c None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:40.313307 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7560bcb4-d250-442f-b0e6-ade1abbf736f None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:40.347998 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cdc8c8-9254-455f-830b-ca1a3c5f06c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:40.348946 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cdc8c8-9254-455f-830b-ca1a3c5f06c2 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:40.349448 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cdc8c8-9254-455f-830b-ca1a3c5f06c2 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:40.349955 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cdc8c8-9254-455f-830b-ca1a3c5f06c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:40.350870 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cdc8c8-9254-455f-830b-ca1a3c5f06c2 None admin] RBAC: Adding query filter params (role.id=14615e30ed3f4d8cb78ee5e0bf9b5761, scope.project.id=5667e30274a94890a1493a76daa4eced, user.id=6a470f28aa544b3cb42de175f4e32d94) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:31:40.351398 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cdc8c8-9254-455f-830b-ca1a3c5f06c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:40.353027 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cdc8c8-9254-455f-830b-ca1a3c5f06c2 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:40.360103 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 63/121] 10.222.0.30 () {56 vars in 1374 bytes} [Sat Oct 15 08:31:40 2022] GET /identity/v3/role_assignments?user.id=6a470f28aa544b3cb42de175f4e32d94&scope.project.id=5667e30274a94890a1493a76daa4eced&role.id=14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:40.370996 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cdc8c8-9254-455f-830b-ca1a3c5f06c2 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:40.385790 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 59/122] 10.222.0.30 () {58 vars in 1282 bytes} [Sat Oct 15 08:31:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 08:31:40.387030 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efa32c6-9b36-43f9-bf9f-f3b1fb94a81c None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:40.450215 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:40.513135 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:40.529522 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efa32c6-9b36-43f9-bf9f-f3b1fb94a81c None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:40.533038 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cdc8c8-9254-455f-830b-ca1a3c5f06c2 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:40.568230 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07428b24-0a6d-4e4a-ba09-9be9d7fe0026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:40.569768 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07428b24-0a6d-4e4a-ba09-9be9d7fe0026 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:40.569768 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07428b24-0a6d-4e4a-ba09-9be9d7fe0026 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:40.569768 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07428b24-0a6d-4e4a-ba09-9be9d7fe0026 None admin] PATH_INFO: `/v3/users/d4cf2957a0f5441cb522c75512551dee` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:40.571680 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07428b24-0a6d-4e4a-ba09-9be9d7fe0026 None admin] RBAC: Authorizing `identity:get_user(user_id=d4cf2957a0f5441cb522c75512551dee)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:40.573022 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8290144f-a134-4777-8a14-7376aca26265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:40.573234 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07428b24-0a6d-4e4a-ba09-9be9d7fe0026 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:40.573708 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8290144f-a134-4777-8a14-7376aca26265 None admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:40.574043 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8290144f-a134-4777-8a14-7376aca26265 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:40.574328 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8290144f-a134-4777-8a14-7376aca26265 None admin] PATH_INFO: `/v3/projects/5667e30274a94890a1493a76daa4eced/users/eeb31007ce8f47d69b196d7bd51f82a7/roles/080a66722a3d4936966a3a50bf9bafe1` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:40.574908 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 60/123] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 08:31:40 2022] GET /identity/v3/users/d4cf2957a0f5441cb522c75512551dee => generated 275 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:40.575773 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07428b24-0a6d-4e4a-ba09-9be9d7fe0026 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:40.581149 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8290144f-a134-4777-8a14-7376aca26265 None admin] RBAC: Authorizing `identity:create_grant(project_id=5667e30274a94890a1493a76daa4eced, user_id=eeb31007ce8f47d69b196d7bd51f82a7, role_id=080a66722a3d4936966a3a50bf9bafe1)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:40.582648 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8290144f-a134-4777-8a14-7376aca26265 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:40.600713 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 64/124] 10.222.0.30 () {58 vars in 1439 bytes} [Sat Oct 15 08:31:40 2022] PUT /identity/v3/projects/5667e30274a94890a1493a76daa4eced/users/eeb31007ce8f47d69b196d7bd51f82a7/roles/080a66722a3d4936966a3a50bf9bafe1 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:31:40.602077 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8290144f-a134-4777-8a14-7376aca26265 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:40.719546 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07428b24-0a6d-4e4a-ba09-9be9d7fe0026 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:40.733751 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:40.735552 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8290144f-a134-4777-8a14-7376aca26265 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:40.762758 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfc59e3-75ca-4a8d-8508-247a0c88459e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:40.763880 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfc59e3-75ca-4a8d-8508-247a0c88459e None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:40.764400 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfc59e3-75ca-4a8d-8508-247a0c88459e None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:40.764881 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfc59e3-75ca-4a8d-8508-247a0c88459e None admin] PATH_INFO: `/v3/projects/5667e30274a94890a1493a76daa4eced` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:40.766493 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfc59e3-75ca-4a8d-8508-247a0c88459e None admin] RBAC: Authorizing `identity:get_project(project_id=5667e30274a94890a1493a76daa4eced)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:40.768535 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfc59e3-75ca-4a8d-8508-247a0c88459e None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:40.771158 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 61/125] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 08:31:40 2022] GET /identity/v3/projects/5667e30274a94890a1493a76daa4eced => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:40.777889 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfc59e3-75ca-4a8d-8508-247a0c88459e None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:40.781485 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d965ee5-8d01-4272-9a9f-f649674c78ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:40.782097 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d965ee5-8d01-4272-9a9f-f649674c78ce None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:40.782395 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d965ee5-8d01-4272-9a9f-f649674c78ce None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:40.782683 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d965ee5-8d01-4272-9a9f-f649674c78ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:40.803070 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:40.858873 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d965ee5-8d01-4272-9a9f-f649674c78ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:40.860402 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d965ee5-8d01-4272-9a9f-f649674c78ce None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:40.892498 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfc59e3-75ca-4a8d-8508-247a0c88459e None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:40.926628 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be453acc-1622-4c46-bf10-439e255b7ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:40.927240 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be453acc-1622-4c46-bf10-439e255b7ef5 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:40.927530 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be453acc-1622-4c46-bf10-439e255b7ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:40.927806 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be453acc-1622-4c46-bf10-439e255b7ef5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:40.928538 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be453acc-1622-4c46-bf10-439e255b7ef5 None admin] RBAC: Adding query filter params (role.id=d236b691c1434103b73cc4fd0b39fec1, scope.project.id=5667e30274a94890a1493a76daa4eced, user.id=d4cf2957a0f5441cb522c75512551dee) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:31:40.928848 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be453acc-1622-4c46-bf10-439e255b7ef5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:40.930337 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be453acc-1622-4c46-bf10-439e255b7ef5 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:40.937393 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 62/126] 10.222.0.30 () {56 vars in 1374 bytes} [Sat Oct 15 08:31:40 2022] GET /identity/v3/role_assignments?user.id=d4cf2957a0f5441cb522c75512551dee&scope.project.id=5667e30274a94890a1493a76daa4eced&role.id=d236b691c1434103b73cc4fd0b39fec1 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:40.938605 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72f88d-d38a-447b-aef5-5b666ce0d6f6 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:40.938890 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72f88d-d38a-447b-aef5-5b666ce0d6f6 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:40.939180 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72f88d-d38a-447b-aef5-5b666ce0d6f6 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:40.940403 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 63/127] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:31:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:31:40.946153 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2837637a-9b66-4cab-8356-64314a57defe None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:40.946445 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2837637a-9b66-4cab-8356-64314a57defe None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:40.946727 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2837637a-9b66-4cab-8356-64314a57defe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:40.966176 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-2837637a-9b66-4cab-8356-64314a57defe None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:31:40.967404 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-2837637a-9b66-4cab-8356-64314a57defe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:31:40.976921 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 65/128] 10.222.0.30 () {58 vars in 1282 bytes} [Sat Oct 15 08:31:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 08:31:40.986979 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d965ee5-8d01-4272-9a9f-f649674c78ce None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:41.017471 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:41.051549 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:41.056444 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 64/129] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:31:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 08:31:41.111901 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d965ee5-8d01-4272-9a9f-f649674c78ce None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:41.150569 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc65203-08dc-44ed-9f50-122c7de758c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:41.151273 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc65203-08dc-44ed-9f50-122c7de758c2 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:41.151564 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc65203-08dc-44ed-9f50-122c7de758c2 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:41.151902 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc65203-08dc-44ed-9f50-122c7de758c2 None admin] PATH_INFO: `/v3/projects/167e11807cf7410b9f0eb9c73d18c927` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:41.152983 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc65203-08dc-44ed-9f50-122c7de758c2 None admin] RBAC: Authorizing `identity:get_project(project_id=167e11807cf7410b9f0eb9c73d18c927)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:41.154482 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc65203-08dc-44ed-9f50-122c7de758c2 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:41.156288 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 66/130] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 08:31:40 2022] GET /identity/v3/projects/167e11807cf7410b9f0eb9c73d18c927 => generated 312 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 08:31:41.162227 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2837637a-9b66-4cab-8356-64314a57defe None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:41.270516 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2837637a-9b66-4cab-8356-64314a57defe None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:41.306938 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f46107b-f88f-42b4-975b-cb2ff074f54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:41.307584 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f46107b-f88f-42b4-975b-cb2ff074f54d None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:41.307873 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f46107b-f88f-42b4-975b-cb2ff074f54d None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:41.308153 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f46107b-f88f-42b4-975b-cb2ff074f54d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:41.308882 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f46107b-f88f-42b4-975b-cb2ff074f54d None admin] RBAC: Adding query filter params (role.id=3248f0268738489b883041851d681bb7, scope.project.id=167e11807cf7410b9f0eb9c73d18c927, user.id=d4cf2957a0f5441cb522c75512551dee) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:31:41.309200 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f46107b-f88f-42b4-975b-cb2ff074f54d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:41.310692 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f46107b-f88f-42b4-975b-cb2ff074f54d None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:41.316582 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 65/131] 10.222.0.30 () {56 vars in 1374 bytes} [Sat Oct 15 08:31:41 2022] GET /identity/v3/role_assignments?user.id=d4cf2957a0f5441cb522c75512551dee&scope.project.id=167e11807cf7410b9f0eb9c73d18c927&role.id=3248f0268738489b883041851d681bb7 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:41.328321 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc65203-08dc-44ed-9f50-122c7de758c2 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:41.443379 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc65203-08dc-44ed-9f50-122c7de758c2 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:41.461955 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:41.482302 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c364cd30-c79c-4243-81ae-30a848b2ce7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:41.482950 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c364cd30-c79c-4243-81ae-30a848b2ce7a None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:41.483234 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c364cd30-c79c-4243-81ae-30a848b2ce7a None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:41.483508 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c364cd30-c79c-4243-81ae-30a848b2ce7a None admin] PATH_INFO: `/v3/domains/default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:41.484613 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c364cd30-c79c-4243-81ae-30a848b2ce7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:41.486532 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c364cd30-c79c-4243-81ae-30a848b2ce7a None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:41.487824 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 67/132] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:31:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:41.493128 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f46107b-f88f-42b4-975b-cb2ff074f54d None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:41.533974 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:41.598695 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f46107b-f88f-42b4-975b-cb2ff074f54d None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:41.631567 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbafa28f-24a9-445b-96bb-3774f93b1523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:41.632174 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbafa28f-24a9-445b-96bb-3774f93b1523 None admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:41.632457 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbafa28f-24a9-445b-96bb-3774f93b1523 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:41.632734 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbafa28f-24a9-445b-96bb-3774f93b1523 None admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:41.633377 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbafa28f-24a9-445b-96bb-3774f93b1523 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:41.634939 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbafa28f-24a9-445b-96bb-3774f93b1523 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:41.660385 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 66/133] 10.222.0.30 () {60 vars in 1132 bytes} [Sat Oct 15 08:31:41 2022] POST /identity/v3/users => generated 283 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 08:31:42.804557 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1944dd-e25f-4569-8c41-f60783c77aaf None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:42.805095 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1944dd-e25f-4569-8c41-f60783c77aaf None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:42.805095 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1944dd-e25f-4569-8c41-f60783c77aaf None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:42.806476 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 68/134] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:31:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:31:42.812393 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1504e379-3552-481f-8adf-8f12a159867b None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:42.812803 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1504e379-3552-481f-8adf-8f12a159867b None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:42.812961 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1504e379-3552-481f-8adf-8f12a159867b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:42.831547 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-1504e379-3552-481f-8adf-8f12a159867b None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:31:42.832719 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-1504e379-3552-481f-8adf-8f12a159867b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:31:42.899386 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 67/135] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:31:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 08:31:43.168959 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1944dd-e25f-4569-8c41-f60783c77aaf None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:43.274187 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1944dd-e25f-4569-8c41-f60783c77aaf None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:43.310321 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24562c92-a3e8-451d-ab7d-9da42b7aa859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:43.311277 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24562c92-a3e8-451d-ab7d-9da42b7aa859 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:43.311761 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24562c92-a3e8-451d-ab7d-9da42b7aa859 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:43.312248 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24562c92-a3e8-451d-ab7d-9da42b7aa859 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:43.322888 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24562c92-a3e8-451d-ab7d-9da42b7aa859 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:31:43.327893 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-24562c92-a3e8-451d-ab7d-9da42b7aa859 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 15 08:31:43.327893 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:31:43.327893 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:31:43.327893 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:31:43.327893 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:31:43.327893 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:31:43.327893 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:31:43.327893 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:31:43.327893 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:31:43.327893 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:31:43.327893 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:31:43.327893 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:31:43.327893 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:31:43.327893 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:31:43.327893 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:31:43.327893 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:31:43.328767 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:31:43.328767 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:31:43.328767 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:31:43.328767 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:31:43.328767 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:31:43.328767 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:31:43.328767 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:31:43.328767 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:31:43.328767 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:31:43.328767 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:31:43.328767 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:31:43.328767 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:31:43.328767 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:31:43.328767 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:31:43.328767 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:31:43.328767 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:31:43.329566 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:31:43.329566 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:31:43.329566 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:31:43.329566 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:31:43.329566 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:31:43.329566 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:31:43.329566 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:31:43.329566 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:31:43.329566 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:31:43.329566 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:31:43.329566 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 15 08:31:43.329566 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:31:43.331774 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 69/136] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 08:31:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:31:43.339113 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1504e379-3552-481f-8adf-8f12a159867b None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:43.363379 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:43.452201 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1504e379-3552-481f-8adf-8f12a159867b None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:43.487520 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3def1a-8e71-4886-991b-d437c706d8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:43.488141 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3def1a-8e71-4886-991b-d437c706d8ce None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:43.488425 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3def1a-8e71-4886-991b-d437c706d8ce None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:43.488704 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3def1a-8e71-4886-991b-d437c706d8ce None admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:43.489379 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3def1a-8e71-4886-991b-d437c706d8ce None admin] RBAC: Adding query filter params (name=reader) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:31:43.489705 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3def1a-8e71-4886-991b-d437c706d8ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:43.491126 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3def1a-8e71-4886-991b-d437c706d8ce None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:43.500071 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 68/137] 10.222.0.30 () {56 vars in 1101 bytes} [Sat Oct 15 08:31:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:43.505869 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24562c92-a3e8-451d-ab7d-9da42b7aa859 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:43.560635 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:43.619121 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24562c92-a3e8-451d-ab7d-9da42b7aa859 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:43.655328 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70abc723-051f-4f0a-a075-400a4ff5ef0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:43.655963 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70abc723-051f-4f0a-a075-400a4ff5ef0b None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:43.656248 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70abc723-051f-4f0a-a075-400a4ff5ef0b None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:43.656527 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70abc723-051f-4f0a-a075-400a4ff5ef0b None admin] PATH_INFO: `/v3/users/eeb31007ce8f47d69b196d7bd51f82a7` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:43.657671 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70abc723-051f-4f0a-a075-400a4ff5ef0b None admin] RBAC: Authorizing `identity:get_user(user_id=eeb31007ce8f47d69b196d7bd51f82a7)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:43.659191 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70abc723-051f-4f0a-a075-400a4ff5ef0b None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:43.660970 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 70/138] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 08:31:43 2022] GET /identity/v3/users/eeb31007ce8f47d69b196d7bd51f82a7 => generated 289 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:43.666384 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3def1a-8e71-4886-991b-d437c706d8ce None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:43.752839 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:43.772352 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3def1a-8e71-4886-991b-d437c706d8ce None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:43.807865 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa8a667-3ff1-420b-a1f6-fc94acb9abb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:43.808486 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa8a667-3ff1-420b-a1f6-fc94acb9abb8 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:43.808776 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa8a667-3ff1-420b-a1f6-fc94acb9abb8 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:43.809059 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa8a667-3ff1-420b-a1f6-fc94acb9abb8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:43.883072 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa8a667-3ff1-420b-a1f6-fc94acb9abb8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:43.884568 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa8a667-3ff1-420b-a1f6-fc94acb9abb8 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:43.970011 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f2b58-7a49-4b92-a9da-00955693f9cc None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:43.970577 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f2b58-7a49-4b92-a9da-00955693f9cc None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:43.970577 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f2b58-7a49-4b92-a9da-00955693f9cc None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:43.972264 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 71/139] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:31:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:31:43.975037 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:43.978339 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105c7ec9-bac5-4089-bad3-dd0312543a34 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:43.978675 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105c7ec9-bac5-4089-bad3-dd0312543a34 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:43.979478 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105c7ec9-bac5-4089-bad3-dd0312543a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:43.997255 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 69/140] 10.222.0.30 () {58 vars in 1282 bytes} [Sat Oct 15 08:31:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 08:31:44.003400 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-105c7ec9-bac5-4089-bad3-dd0312543a34 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:31:44.003664 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa8a667-3ff1-420b-a1f6-fc94acb9abb8 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:44.004546 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-105c7ec9-bac5-4089-bad3-dd0312543a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:31:44.075736 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 72/141] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:31:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 08:31:44.090366 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c733851-e10b-4fea-91bd-dce678f6fd0a None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:44.090715 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c733851-e10b-4fea-91bd-dce678f6fd0a None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:44.091003 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c733851-e10b-4fea-91bd-dce678f6fd0a None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:44.092590 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 73/142] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:31:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:31:44.099012 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6100db48-1869-49dd-91ae-a919d7b13cf9 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:44.099861 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6100db48-1869-49dd-91ae-a919d7b13cf9 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:44.100204 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6100db48-1869-49dd-91ae-a919d7b13cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:44.115367 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa8a667-3ff1-420b-a1f6-fc94acb9abb8 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:44.121315 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-6100db48-1869-49dd-91ae-a919d7b13cf9 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:31:44.122531 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-6100db48-1869-49dd-91ae-a919d7b13cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:31:44.152350 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699c664c-b703-4c14-99e4-3070efc6a8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:44.153172 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699c664c-b703-4c14-99e4-3070efc6a8cf None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:44.153609 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699c664c-b703-4c14-99e4-3070efc6a8cf None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:44.154058 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699c664c-b703-4c14-99e4-3070efc6a8cf None admin] PATH_INFO: `/v3/projects/5667e30274a94890a1493a76daa4eced` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:44.155297 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699c664c-b703-4c14-99e4-3070efc6a8cf None admin] RBAC: Authorizing `identity:get_project(project_id=5667e30274a94890a1493a76daa4eced)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:44.157122 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699c664c-b703-4c14-99e4-3070efc6a8cf None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:44.157520 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:44.159603 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 70/143] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 08:31:44 2022] GET /identity/v3/projects/5667e30274a94890a1493a76daa4eced => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 08:31:44.160724 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c106b-1a07-4b8d-90b1-e8103ff2fb41 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:44.161004 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c106b-1a07-4b8d-90b1-e8103ff2fb41 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:44.161279 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c106b-1a07-4b8d-90b1-e8103ff2fb41 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:44.162475 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 71/144] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:31:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:31:44.167102 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3c106b-1a07-4b8d-90b1-e8103ff2fb41 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:44.193421 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 74/145] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:31:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 08:31:44.201871 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58c6bb9-7a1a-4324-85dd-00d456e0ecda None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:44.201871 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58c6bb9-7a1a-4324-85dd-00d456e0ecda None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:44.201871 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58c6bb9-7a1a-4324-85dd-00d456e0ecda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:44.223988 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-e58c6bb9-7a1a-4324-85dd-00d456e0ecda None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:31:44.224481 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:44.225714 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-e58c6bb9-7a1a-4324-85dd-00d456e0ecda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:31:44.269466 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3c106b-1a07-4b8d-90b1-e8103ff2fb41 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:44.303333 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473bba10-fa78-4a79-ae31-37a5574c5303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:44.304091 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473bba10-fa78-4a79-ae31-37a5574c5303 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:44.304477 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473bba10-fa78-4a79-ae31-37a5574c5303 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:44.304858 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473bba10-fa78-4a79-ae31-37a5574c5303 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:44.305710 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473bba10-fa78-4a79-ae31-37a5574c5303 None admin] RBAC: Adding query filter params (role.id=080a66722a3d4936966a3a50bf9bafe1, scope.project.id=5667e30274a94890a1493a76daa4eced, user.id=eeb31007ce8f47d69b196d7bd51f82a7) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:31:44.306141 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473bba10-fa78-4a79-ae31-37a5574c5303 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:44.308213 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473bba10-fa78-4a79-ae31-37a5574c5303 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:44.322063 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 75/146] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:31:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 08:31:44.327750 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 72/147] 10.222.0.30 () {56 vars in 1374 bytes} [Sat Oct 15 08:31:44 2022] GET /identity/v3/role_assignments?user.id=eeb31007ce8f47d69b196d7bd51f82a7&scope.project.id=5667e30274a94890a1493a76daa4eced&role.id=080a66722a3d4936966a3a50bf9bafe1 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:44.349810 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58c6bb9-7a1a-4324-85dd-00d456e0ecda None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:44.462587 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58c6bb9-7a1a-4324-85dd-00d456e0ecda None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:44.464888 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:44.469928 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473bba10-fa78-4a79-ae31-37a5574c5303 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:44.507682 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721f1f8d-ff28-4236-8ac6-b5c93c196361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:44.508321 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f1f8d-ff28-4236-8ac6-b5c93c196361 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:44.508759 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f1f8d-ff28-4236-8ac6-b5c93c196361 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:44.509040 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f1f8d-ff28-4236-8ac6-b5c93c196361 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:44.525106 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721f1f8d-ff28-4236-8ac6-b5c93c196361 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:31:44.530676 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-721f1f8d-ff28-4236-8ac6-b5c93c196361 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 08:31:44.530676 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:31:44.530676 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:31:44.530676 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:31:44.530676 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:31:44.530676 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:31:44.530676 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:31:44.530676 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:31:44.530676 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:31:44.530676 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:31:44.530676 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:31:44.530676 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:31:44.530676 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:31:44.530676 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:31:44.530676 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:31:44.530676 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:31:44.531512 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:31:44.531512 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:31:44.531512 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:31:44.531512 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:31:44.531512 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:31:44.531512 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:31:44.531512 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:31:44.531512 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:31:44.531512 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:31:44.531512 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:31:44.531512 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:31:44.531512 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:31:44.531512 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:31:44.531512 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:31:44.531512 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:31:44.531512 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:31:44.532423 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:31:44.532423 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:31:44.532423 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:31:44.532423 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:31:44.532423 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:31:44.532423 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:31:44.532423 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:31:44.532423 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:31:44.532423 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:31:44.532423 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:31:44.532423 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 08:31:44.532423 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:31:44.532423 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 76/148] 10.222.0.30 () {56 vars in 1096 bytes} [Sat Oct 15 08:31:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:31:44.537793 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721f1f8d-ff28-4236-8ac6-b5c93c196361 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:44.595016 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473bba10-fa78-4a79-ae31-37a5574c5303 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:44.613863 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:44.633423 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24efc3d8-3243-4823-9b8d-c2b60afabb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:44.634165 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24efc3d8-3243-4823-9b8d-c2b60afabb94 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:44.634451 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24efc3d8-3243-4823-9b8d-c2b60afabb94 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:44.634731 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24efc3d8-3243-4823-9b8d-c2b60afabb94 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:44.636540 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24efc3d8-3243-4823-9b8d-c2b60afabb94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:44.638044 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24efc3d8-3243-4823-9b8d-c2b60afabb94 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:44.639794 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 73/149] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:31:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 08:31:44.640411 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24efc3d8-3243-4823-9b8d-c2b60afabb94 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:44.653933 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721f1f8d-ff28-4236-8ac6-b5c93c196361 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:44.689104 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d362c8-3583-4c95-a135-3eee242785ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:44.689748 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d362c8-3583-4c95-a135-3eee242785ba None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:44.690037 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d362c8-3583-4c95-a135-3eee242785ba None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:44.690313 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d362c8-3583-4c95-a135-3eee242785ba None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:44.690975 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d362c8-3583-4c95-a135-3eee242785ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:31:44.691572 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d362c8-3583-4c95-a135-3eee242785ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:44.692946 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d362c8-3583-4c95-a135-3eee242785ba None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:44.700840 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 77/150] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 08:31:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 08:31:44.705928 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d362c8-3583-4c95-a135-3eee242785ba None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:44.730642 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:44.739427 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24efc3d8-3243-4823-9b8d-c2b60afabb94 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:44.772453 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfc0ff8-40b7-4cb8-87f8-1530370692c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:44.773307 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfc0ff8-40b7-4cb8-87f8-1530370692c9 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:44.773587 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfc0ff8-40b7-4cb8-87f8-1530370692c9 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:44.773889 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfc0ff8-40b7-4cb8-87f8-1530370692c9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:44.784063 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfc0ff8-40b7-4cb8-87f8-1530370692c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:31:44.788775 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-0cfc0ff8-40b7-4cb8-87f8-1530370692c9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 08:31:44.788775 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:31:44.788775 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:31:44.788775 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:31:44.788775 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:31:44.788775 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:31:44.788775 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:31:44.788775 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:31:44.788775 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:31:44.788775 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:31:44.788775 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:31:44.788775 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:31:44.788775 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:31:44.788775 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:31:44.788775 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:31:44.788775 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:31:44.789633 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:31:44.789633 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:31:44.789633 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:31:44.789633 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:31:44.789633 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:31:44.789633 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:31:44.789633 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:31:44.789633 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:31:44.789633 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:31:44.789633 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:31:44.789633 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:31:44.789633 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:31:44.789633 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:31:44.789633 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:31:44.789633 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:31:44.789633 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:31:44.790614 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:31:44.790614 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:31:44.790614 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:31:44.790614 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:31:44.790614 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:31:44.790614 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:31:44.790614 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:31:44.790614 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:31:44.790614 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:31:44.790614 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:31:44.790614 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 08:31:44.790614 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:31:44.790614 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 74/151] 10.222.0.30 () {56 vars in 1114 bytes} [Sat Oct 15 08:31:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:31:44.792429 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfc0ff8-40b7-4cb8-87f8-1530370692c9 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:44.811573 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d362c8-3583-4c95-a135-3eee242785ba None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:44.838639 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:44.844596 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056bd86b-5d4a-48b6-8964-22859b060878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:44.845205 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056bd86b-5d4a-48b6-8964-22859b060878 None admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:44.845487 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056bd86b-5d4a-48b6-8964-22859b060878 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:44.845794 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056bd86b-5d4a-48b6-8964-22859b060878 None admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:44.846444 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056bd86b-5d4a-48b6-8964-22859b060878 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:44.847887 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056bd86b-5d4a-48b6-8964-22859b060878 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:44.869237 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 78/152] 10.222.0.30 () {60 vars in 1132 bytes} [Sat Oct 15 08:31:44 2022] POST /identity/v3/users => generated 297 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:31:44.870166 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056bd86b-5d4a-48b6-8964-22859b060878 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:44.916465 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfc0ff8-40b7-4cb8-87f8-1530370692c9 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:44.957005 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51cc492-2c61-48dd-aa87-9c67c1cdd4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:44.957795 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51cc492-2c61-48dd-aa87-9c67c1cdd4d9 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:44.958081 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51cc492-2c61-48dd-aa87-9c67c1cdd4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:44.958365 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51cc492-2c61-48dd-aa87-9c67c1cdd4d9 None admin] PATH_INFO: `/v3/users/5ab448f9598c4699a79802391e4d7baa` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:44.960705 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:44.978894 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51cc492-2c61-48dd-aa87-9c67c1cdd4d9 None admin] RBAC: Authorizing `identity:get_user(user_id=5ab448f9598c4699a79802391e4d7baa)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:44.981405 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51cc492-2c61-48dd-aa87-9c67c1cdd4d9 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:44.982336 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056bd86b-5d4a-48b6-8964-22859b060878 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:44.984225 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 75/153] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 08:31:44 2022] GET /identity/v3/users/5ab448f9598c4699a79802391e4d7baa => generated 283 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:44.990062 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51cc492-2c61-48dd-aa87-9c67c1cdd4d9 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:45.023589 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89c6bea-c3d7-4168-aeb4-fe7692a94e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:45.024220 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c6bea-c3d7-4168-aeb4-fe7692a94e13 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:45.024502 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c6bea-c3d7-4168-aeb4-fe7692a94e13 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:45.024774 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c6bea-c3d7-4168-aeb4-fe7692a94e13 None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:45.025462 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89c6bea-c3d7-4168-aeb4-fe7692a94e13 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:31:45.026298 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89c6bea-c3d7-4168-aeb4-fe7692a94e13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:45.027721 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89c6bea-c3d7-4168-aeb4-fe7692a94e13 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:45.037671 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 79/154] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 08:31:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:45.043292 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89c6bea-c3d7-4168-aeb4-fe7692a94e13 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:45.109443 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51cc492-2c61-48dd-aa87-9c67c1cdd4d9 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:45.112631 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:45.150489 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3164d03c-4f29-42c6-93af-447880377138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:45.151172 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3164d03c-4f29-42c6-93af-447880377138 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:45.151457 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3164d03c-4f29-42c6-93af-447880377138 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:45.151768 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3164d03c-4f29-42c6-93af-447880377138 None admin] PATH_INFO: `/v3/projects/7537ad5b7fef46308a69ce89f36209c2` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:45.153747 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3164d03c-4f29-42c6-93af-447880377138 None admin] RBAC: Authorizing `identity:get_project(project_id=7537ad5b7fef46308a69ce89f36209c2)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:45.155230 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3164d03c-4f29-42c6-93af-447880377138 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:45.157113 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 76/155] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 08:31:44 2022] GET /identity/v3/projects/7537ad5b7fef46308a69ce89f36209c2 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:45.162647 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3164d03c-4f29-42c6-93af-447880377138 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:45.168954 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89c6bea-c3d7-4168-aeb4-fe7692a94e13 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:45.214512 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18346b36-3068-451a-b4a2-9a5106824457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:45.214935 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18346b36-3068-451a-b4a2-9a5106824457 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:45.215412 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18346b36-3068-451a-b4a2-9a5106824457 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:45.215790 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18346b36-3068-451a-b4a2-9a5106824457 None admin] PATH_INFO: `/v3/users/5ab448f9598c4699a79802391e4d7baa` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:45.217001 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18346b36-3068-451a-b4a2-9a5106824457 None admin] RBAC: Authorizing `identity:get_user(user_id=5ab448f9598c4699a79802391e4d7baa)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:45.218571 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18346b36-3068-451a-b4a2-9a5106824457 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:45.220359 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 80/156] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 08:31:45 2022] GET /identity/v3/users/5ab448f9598c4699a79802391e4d7baa => generated 283 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:45.225945 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18346b36-3068-451a-b4a2-9a5106824457 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:45.240117 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:45.287571 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3164d03c-4f29-42c6-93af-447880377138 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:45.327929 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac636f27-a30d-4b11-89cb-4ee14612f1e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:45.328934 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac636f27-a30d-4b11-89cb-4ee14612f1e0 None admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:45.329436 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac636f27-a30d-4b11-89cb-4ee14612f1e0 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:45.329962 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac636f27-a30d-4b11-89cb-4ee14612f1e0 None admin] PATH_INFO: `/v3/projects/7537ad5b7fef46308a69ce89f36209c2/users/5ab448f9598c4699a79802391e4d7baa/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:45.332320 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac636f27-a30d-4b11-89cb-4ee14612f1e0 None admin] RBAC: Authorizing `identity:create_grant(project_id=7537ad5b7fef46308a69ce89f36209c2, user_id=5ab448f9598c4699a79802391e4d7baa, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:45.334115 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac636f27-a30d-4b11-89cb-4ee14612f1e0 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:45.352755 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 77/157] 10.222.0.30 () {58 vars in 1439 bytes} [Sat Oct 15 08:31:45 2022] PUT /identity/v3/projects/7537ad5b7fef46308a69ce89f36209c2/users/5ab448f9598c4699a79802391e4d7baa/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 08:31:45.370262 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18346b36-3068-451a-b4a2-9a5106824457 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:45.422321 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323ec714-6c6b-4eb9-a7b3-ecf399ee176c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:45.423396 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ec714-6c6b-4eb9-a7b3-ecf399ee176c None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:45.423887 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ec714-6c6b-4eb9-a7b3-ecf399ee176c None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:45.424380 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ec714-6c6b-4eb9-a7b3-ecf399ee176c None admin] PATH_INFO: `/v3/projects/7537ad5b7fef46308a69ce89f36209c2` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:45.425973 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323ec714-6c6b-4eb9-a7b3-ecf399ee176c None admin] RBAC: Authorizing `identity:get_project(project_id=7537ad5b7fef46308a69ce89f36209c2)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:45.427728 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323ec714-6c6b-4eb9-a7b3-ecf399ee176c None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:45.430094 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 81/158] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 08:31:45 2022] GET /identity/v3/projects/7537ad5b7fef46308a69ce89f36209c2 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 08:31:45.439037 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac636f27-a30d-4b11-89cb-4ee14612f1e0 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:45.480062 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:45.571630 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac636f27-a30d-4b11-89cb-4ee14612f1e0 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:45.608043 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b279809a-ae2b-47bd-b1bc-169cbeaead74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:45.608632 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279809a-ae2b-47bd-b1bc-169cbeaead74 None admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:45.608911 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279809a-ae2b-47bd-b1bc-169cbeaead74 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:45.609185 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279809a-ae2b-47bd-b1bc-169cbeaead74 None admin] PATH_INFO: `/v3/projects/7537ad5b7fef46308a69ce89f36209c2/users/5ab448f9598c4699a79802391e4d7baa/roles/d236b691c1434103b73cc4fd0b39fec1` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:45.611371 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b279809a-ae2b-47bd-b1bc-169cbeaead74 None admin] RBAC: Authorizing `identity:create_grant(project_id=7537ad5b7fef46308a69ce89f36209c2, user_id=5ab448f9598c4699a79802391e4d7baa, role_id=d236b691c1434103b73cc4fd0b39fec1)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:45.612817 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b279809a-ae2b-47bd-b1bc-169cbeaead74 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:45.627112 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 78/159] 10.222.0.30 () {58 vars in 1439 bytes} [Sat Oct 15 08:31:45 2022] PUT /identity/v3/projects/7537ad5b7fef46308a69ce89f36209c2/users/5ab448f9598c4699a79802391e4d7baa/roles/d236b691c1434103b73cc4fd0b39fec1 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:31:47.264255 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656bcf32-fdfc-45d4-b884-a0301217f660 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:47.264781 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656bcf32-fdfc-45d4-b884-a0301217f660 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:47.264856 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656bcf32-fdfc-45d4-b884-a0301217f660 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:47.266432 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 82/160] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:31:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:31:47.272405 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93bd177-b560-4510-986d-760444bccc72 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:47.272930 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93bd177-b560-4510-986d-760444bccc72 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:47.273004 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93bd177-b560-4510-986d-760444bccc72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:47.294810 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-a93bd177-b560-4510-986d-760444bccc72 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:31:47.296030 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-a93bd177-b560-4510-986d-760444bccc72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:31:47.375305 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 79/161] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:31:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 08:31:47.406040 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfddf00a-8c7e-4428-b9b2-42d8d171ed0e None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:47.406333 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfddf00a-8c7e-4428-b9b2-42d8d171ed0e None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:47.406617 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfddf00a-8c7e-4428-b9b2-42d8d171ed0e None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:47.408097 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 83/162] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:31:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:31:47.413367 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7803c097-5288-4fdb-8633-a87736acd5a9 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:47.413654 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7803c097-5288-4fdb-8633-a87736acd5a9 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:47.413977 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7803c097-5288-4fdb-8633-a87736acd5a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:47.432978 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-7803c097-5288-4fdb-8633-a87736acd5a9 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:31:47.434203 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-7803c097-5288-4fdb-8633-a87736acd5a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:31:47.455100 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:47.514792 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 80/163] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:31:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 08:31:47.660681 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfddf00a-8c7e-4428-b9b2-42d8d171ed0e None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:47.671891 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b1d6f-07a9-451f-968d-2a8cd3e9b62c None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:47.672180 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b1d6f-07a9-451f-968d-2a8cd3e9b62c None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:47.672462 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b1d6f-07a9-451f-968d-2a8cd3e9b62c None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:47.674008 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 81/164] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:31:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:31:47.680727 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba2497c-3512-468f-9bdc-4d10cd8969fa None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:47.681020 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba2497c-3512-468f-9bdc-4d10cd8969fa None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:47.681302 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba2497c-3512-468f-9bdc-4d10cd8969fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:47.706615 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-eba2497c-3512-468f-9bdc-4d10cd8969fa None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:31:47.707824 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-eba2497c-3512-468f-9bdc-4d10cd8969fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:31:47.763694 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:47.794955 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfddf00a-8c7e-4428-b9b2-42d8d171ed0e None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:47.808630 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 82/165] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:31:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 08:31:47.809963 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba2497c-3512-468f-9bdc-4d10cd8969fa None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:47.839872 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d46a40f-442f-412f-9ca8-7f739d939cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:47.840506 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d46a40f-442f-412f-9ca8-7f739d939cd0 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:47.840918 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d46a40f-442f-412f-9ca8-7f739d939cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:47.841161 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d46a40f-442f-412f-9ca8-7f739d939cd0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:47.854262 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d46a40f-442f-412f-9ca8-7f739d939cd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:31:47.859435 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-6d46a40f-442f-412f-9ca8-7f739d939cd0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 15 08:31:47.859435 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:31:47.859435 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:31:47.859435 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:31:47.859435 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:31:47.859435 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:31:47.859435 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:31:47.859435 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:31:47.859435 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:31:47.859435 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:31:47.859435 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:31:47.859435 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:31:47.859435 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:31:47.859435 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:31:47.859435 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:31:47.859435 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:31:47.860339 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:31:47.860339 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:31:47.860339 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:31:47.860339 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:31:47.860339 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:31:47.860339 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:31:47.860339 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:31:47.860339 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:31:47.860339 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:31:47.860339 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:31:47.860339 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:31:47.860339 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:31:47.860339 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:31:47.860339 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:31:47.860339 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:31:47.860339 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:31:47.861335 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:31:47.861335 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:31:47.861335 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:31:47.861335 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:31:47.861335 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:31:47.861335 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:31:47.861335 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:31:47.861335 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:31:47.861335 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:31:47.861335 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:31:47.861335 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 15 08:31:47.861335 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:31:47.861335 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 84/166] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 08:31:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:31:47.866407 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d46a40f-442f-412f-9ca8-7f739d939cd0 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:47.905355 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:47.926839 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba2497c-3512-468f-9bdc-4d10cd8969fa None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:47.962413 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af655ad6-5f8d-4ff1-b1cd-c2e309f7df16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:47.963714 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af655ad6-5f8d-4ff1-b1cd-c2e309f7df16 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:47.964001 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af655ad6-5f8d-4ff1-b1cd-c2e309f7df16 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:47.964392 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af655ad6-5f8d-4ff1-b1cd-c2e309f7df16 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:47.965508 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af655ad6-5f8d-4ff1-b1cd-c2e309f7df16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:47.967128 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af655ad6-5f8d-4ff1-b1cd-c2e309f7df16 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:47.968964 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 83/167] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:31:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 08:31:47.975736 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53e11fc-ec55-4136-a4c4-398addc25b4b None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:47.975736 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53e11fc-ec55-4136-a4c4-398addc25b4b None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:47.975736 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53e11fc-ec55-4136-a4c4-398addc25b4b None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:47.976106 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d46a40f-442f-412f-9ca8-7f739d939cd0 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:47.976181 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 84/168] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:31:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:31:47.976181 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53e11fc-ec55-4136-a4c4-398addc25b4b None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:48.011711 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8501f77-bd70-4f7c-bd3d-331e7e2ffb84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:48.012318 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8501f77-bd70-4f7c-bd3d-331e7e2ffb84 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:48.012599 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8501f77-bd70-4f7c-bd3d-331e7e2ffb84 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:48.012875 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8501f77-bd70-4f7c-bd3d-331e7e2ffb84 None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:48.013560 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8501f77-bd70-4f7c-bd3d-331e7e2ffb84 None admin] RBAC: Adding query filter params (name=member) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:31:48.013887 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8501f77-bd70-4f7c-bd3d-331e7e2ffb84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:48.015334 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8501f77-bd70-4f7c-bd3d-331e7e2ffb84 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:48.020943 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:48.027120 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 85/169] 10.222.0.30 () {56 vars in 1101 bytes} [Sat Oct 15 08:31:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 08:31:48.032775 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca7358-b955-4ebe-a356-3ff61812aad3 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:48.033099 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca7358-b955-4ebe-a356-3ff61812aad3 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:48.033633 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca7358-b955-4ebe-a356-3ff61812aad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:48.051199 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-b0ca7358-b955-4ebe-a356-3ff61812aad3 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:31:48.052309 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-b0ca7358-b955-4ebe-a356-3ff61812aad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:31:48.084119 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53e11fc-ec55-4136-a4c4-398addc25b4b None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:48.118703 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b186d24-1da2-4cd4-9b6d-c1e210183f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:48.119818 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 86/170] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:31:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 08:31:48.119929 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b186d24-1da2-4cd4-9b6d-c1e210183f86 None admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:48.120108 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b186d24-1da2-4cd4-9b6d-c1e210183f86 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:48.120379 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b186d24-1da2-4cd4-9b6d-c1e210183f86 None admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:48.120470 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca7358-b955-4ebe-a356-3ff61812aad3 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:48.121037 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b186d24-1da2-4cd4-9b6d-c1e210183f86 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:48.122506 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b186d24-1da2-4cd4-9b6d-c1e210183f86 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:48.144419 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 85/171] 10.222.0.30 () {60 vars in 1132 bytes} [Sat Oct 15 08:31:47 2022] POST /identity/v3/users => generated 297 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 08:31:48.145584 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b186d24-1da2-4cd4-9b6d-c1e210183f86 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:48.159586 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:48.279764 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b186d24-1da2-4cd4-9b6d-c1e210183f86 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:48.285333 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca7358-b955-4ebe-a356-3ff61812aad3 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:48.319678 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:48.327566 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa26a8d-c77e-434c-af47-5057d1772a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:48.328521 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa26a8d-c77e-434c-af47-5057d1772a25 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:48.329019 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa26a8d-c77e-434c-af47-5057d1772a25 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:48.329452 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa26a8d-c77e-434c-af47-5057d1772a25 None admin] PATH_INFO: `/v3/users/299f17920261414a905a73845424d7fa` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:48.330102 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a0677b-bc38-4adf-a01f-1d290bc3bf82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:48.330833 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0677b-bc38-4adf-a01f-1d290bc3bf82 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:48.331214 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0677b-bc38-4adf-a01f-1d290bc3bf82 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:48.331592 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0677b-bc38-4adf-a01f-1d290bc3bf82 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:48.347198 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa26a8d-c77e-434c-af47-5057d1772a25 None admin] RBAC: Authorizing `identity:get_user(user_id=299f17920261414a905a73845424d7fa)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:48.348594 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a0677b-bc38-4adf-a01f-1d290bc3bf82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:31:48.348819 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa26a8d-c77e-434c-af47-5057d1772a25 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:48.350728 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 87/172] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 08:31:48 2022] GET /identity/v3/users/299f17920261414a905a73845424d7fa => generated 297 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:48.354224 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-64a0677b-bc38-4adf-a01f-1d290bc3bf82 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 08:31:48.354224 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:31:48.354224 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:31:48.354224 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:31:48.354224 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:31:48.354224 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:31:48.354224 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:31:48.354224 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:31:48.354224 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:31:48.354224 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:31:48.354224 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:31:48.354224 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:31:48.354224 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:31:48.354224 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:31:48.354224 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:31:48.354224 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:31:48.355085 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:31:48.355085 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:31:48.355085 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:31:48.355085 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:31:48.355085 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:31:48.355085 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:31:48.355085 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:31:48.355085 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:31:48.355085 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:31:48.355085 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:31:48.355085 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:31:48.355085 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:31:48.355085 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:31:48.355085 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:31:48.355085 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:31:48.355085 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:31:48.358794 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:31:48.358794 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:31:48.358794 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:31:48.358794 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:31:48.358794 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:31:48.358794 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:31:48.358794 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:31:48.358794 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:31:48.358794 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:31:48.358794 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:31:48.358794 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 08:31:48.358794 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:31:48.358794 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 86/173] 10.222.0.30 () {56 vars in 1096 bytes} [Sat Oct 15 08:31:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:31:48.359457 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa26a8d-c77e-434c-af47-5057d1772a25 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:48.364811 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a0677b-bc38-4adf-a01f-1d290bc3bf82 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:48.442566 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:48.487384 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa26a8d-c77e-434c-af47-5057d1772a25 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:48.489897 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a0677b-bc38-4adf-a01f-1d290bc3bf82 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:48.532012 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078d8a70-aa94-4c32-9194-52ebd40a072b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:48.532459 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c25451-9dbd-4191-8bf4-5897c1f09a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:48.532656 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078d8a70-aa94-4c32-9194-52ebd40a072b None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:48.532920 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078d8a70-aa94-4c32-9194-52ebd40a072b None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:48.533192 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078d8a70-aa94-4c32-9194-52ebd40a072b None admin] PATH_INFO: `/v3/projects/7537ad5b7fef46308a69ce89f36209c2` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:48.533862 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c25451-9dbd-4191-8bf4-5897c1f09a85 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:48.534320 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078d8a70-aa94-4c32-9194-52ebd40a072b None admin] RBAC: Authorizing `identity:get_project(project_id=7537ad5b7fef46308a69ce89f36209c2)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:48.534440 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c25451-9dbd-4191-8bf4-5897c1f09a85 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:48.534873 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c25451-9dbd-4191-8bf4-5897c1f09a85 None admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:48.535744 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078d8a70-aa94-4c32-9194-52ebd40a072b None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:48.535957 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c25451-9dbd-4191-8bf4-5897c1f09a85 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:31:48.536417 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c25451-9dbd-4191-8bf4-5897c1f09a85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:48.538059 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c25451-9dbd-4191-8bf4-5897c1f09a85 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:48.542905 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 88/174] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 08:31:48 2022] GET /identity/v3/projects/7537ad5b7fef46308a69ce89f36209c2 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:48.543783 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078d8a70-aa94-4c32-9194-52ebd40a072b None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:48.549795 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 87/175] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 08:31:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 08:31:48.550903 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c25451-9dbd-4191-8bf4-5897c1f09a85 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:48.606565 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:48.668203 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078d8a70-aa94-4c32-9194-52ebd40a072b None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:48.679790 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c25451-9dbd-4191-8bf4-5897c1f09a85 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:48.705481 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f2235e-894f-4118-9814-f82788bc627b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:48.706143 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f2235e-894f-4118-9814-f82788bc627b None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:48.706430 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f2235e-894f-4118-9814-f82788bc627b None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:48.706708 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f2235e-894f-4118-9814-f82788bc627b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:48.707321 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:48.718188 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3da0fe-94e4-46f7-a6e5-d636bb9d20be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:48.719151 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3da0fe-94e4-46f7-a6e5-d636bb9d20be None admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:48.719635 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3da0fe-94e4-46f7-a6e5-d636bb9d20be None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:48.719747 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f2235e-894f-4118-9814-f82788bc627b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:31:48.720187 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3da0fe-94e4-46f7-a6e5-d636bb9d20be None admin] PATH_INFO: `/v3/projects/7537ad5b7fef46308a69ce89f36209c2/users/299f17920261414a905a73845424d7fa/roles/3248f0268738489b883041851d681bb7` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:48.722326 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3da0fe-94e4-46f7-a6e5-d636bb9d20be None admin] RBAC: Authorizing `identity:create_grant(project_id=7537ad5b7fef46308a69ce89f36209c2, user_id=299f17920261414a905a73845424d7fa, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:48.723966 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3da0fe-94e4-46f7-a6e5-d636bb9d20be None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:48.725283 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-82f2235e-894f-4118-9814-f82788bc627b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 08:31:48.725283 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:31:48.725283 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:31:48.725283 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:31:48.725283 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:31:48.725283 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:31:48.725283 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:31:48.725283 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:31:48.725283 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:31:48.725283 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:31:48.725283 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:31:48.725283 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:31:48.725283 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:31:48.725283 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:31:48.725283 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:31:48.725283 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:31:48.726270 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:31:48.726270 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:31:48.726270 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:31:48.726270 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:31:48.726270 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:31:48.726270 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:31:48.726270 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:31:48.726270 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:31:48.726270 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:31:48.726270 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:31:48.726270 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:31:48.726270 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:31:48.726270 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:31:48.726270 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:31:48.726270 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:31:48.726270 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:31:48.727242 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:31:48.727242 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:31:48.727242 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:31:48.727242 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:31:48.727242 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:31:48.727242 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:31:48.727242 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:31:48.727242 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:31:48.727242 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:31:48.727242 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:31:48.727242 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 08:31:48.727242 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:31:48.727242 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 89/176] 10.222.0.30 () {56 vars in 1114 bytes} [Sat Oct 15 08:31:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:31:48.727242 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f2235e-894f-4118-9814-f82788bc627b None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:48.751142 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 88/177] 10.222.0.30 () {58 vars in 1439 bytes} [Sat Oct 15 08:31:48 2022] PUT /identity/v3/projects/7537ad5b7fef46308a69ce89f36209c2/users/299f17920261414a905a73845424d7fa/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:31:48.752313 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3da0fe-94e4-46f7-a6e5-d636bb9d20be None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:48.929188 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3da0fe-94e4-46f7-a6e5-d636bb9d20be None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:48.934011 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f2235e-894f-4118-9814-f82788bc627b None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:48.966499 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:48.989406 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de90c3c6-d2c8-42d8-92fc-0b7410422934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:48.990182 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de90c3c6-d2c8-42d8-92fc-0b7410422934 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:48.990472 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de90c3c6-d2c8-42d8-92fc-0b7410422934 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:48.990753 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de90c3c6-d2c8-42d8-92fc-0b7410422934 None admin] PATH_INFO: `/v3/users/5ab448f9598c4699a79802391e4d7baa` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:48.992182 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de90c3c6-d2c8-42d8-92fc-0b7410422934 None admin] RBAC: Authorizing `identity:get_user(user_id=5ab448f9598c4699a79802391e4d7baa)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:48.993255 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d202602d-3c45-48c8-8d97-a3c3aacdafda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:48.993786 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de90c3c6-d2c8-42d8-92fc-0b7410422934 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:48.994894 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d202602d-3c45-48c8-8d97-a3c3aacdafda None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:48.995364 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d202602d-3c45-48c8-8d97-a3c3aacdafda None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:48.995823 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d202602d-3c45-48c8-8d97-a3c3aacdafda None admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:48.996278 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 90/178] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 08:31:48 2022] GET /identity/v3/users/5ab448f9598c4699a79802391e4d7baa => generated 283 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:48.996882 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d202602d-3c45-48c8-8d97-a3c3aacdafda None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:31:48.997395 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d202602d-3c45-48c8-8d97-a3c3aacdafda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:48.999223 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d202602d-3c45-48c8-8d97-a3c3aacdafda None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:49.001811 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de90c3c6-d2c8-42d8-92fc-0b7410422934 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:49.009608 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 89/179] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 08:31:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:49.016267 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d202602d-3c45-48c8-8d97-a3c3aacdafda None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:49.091952 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:49.166655 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de90c3c6-d2c8-42d8-92fc-0b7410422934 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:49.183367 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d202602d-3c45-48c8-8d97-a3c3aacdafda None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:49.226537 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7049fa-6fc8-4351-9670-ab6fd597173f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:49.227172 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7049fa-6fc8-4351-9670-ab6fd597173f None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:49.227454 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7049fa-6fc8-4351-9670-ab6fd597173f None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:49.227728 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7049fa-6fc8-4351-9670-ab6fd597173f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:49.230945 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb9a6ae-fc26-41be-820e-3b8c0b12abd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:49.231764 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb9a6ae-fc26-41be-820e-3b8c0b12abd5 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:49.232148 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb9a6ae-fc26-41be-820e-3b8c0b12abd5 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:49.232529 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb9a6ae-fc26-41be-820e-3b8c0b12abd5 None admin] PATH_INFO: `/v3/users/5ab448f9598c4699a79802391e4d7baa` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:49.233944 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb9a6ae-fc26-41be-820e-3b8c0b12abd5 None admin] RBAC: Authorizing `identity:get_user(user_id=5ab448f9598c4699a79802391e4d7baa)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:49.235597 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb9a6ae-fc26-41be-820e-3b8c0b12abd5 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:49.237439 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 90/180] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 08:31:49 2022] GET /identity/v3/users/5ab448f9598c4699a79802391e4d7baa => generated 283 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:49.243034 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb9a6ae-fc26-41be-820e-3b8c0b12abd5 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:49.330308 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:49.348907 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7049fa-6fc8-4351-9670-ab6fd597173f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:49.350419 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7049fa-6fc8-4351-9670-ab6fd597173f None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:49.460953 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb9a6ae-fc26-41be-820e-3b8c0b12abd5 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:49.469643 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 91/181] 10.222.0.30 () {58 vars in 1282 bytes} [Sat Oct 15 08:31:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 08:31:49.474902 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7049fa-6fc8-4351-9670-ab6fd597173f None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:49.501567 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:49.539428 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3170499c-61e3-486b-99cc-bf3f36a14347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:49.540527 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3170499c-61e3-486b-99cc-bf3f36a14347 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:49.541018 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3170499c-61e3-486b-99cc-bf3f36a14347 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:49.542068 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3170499c-61e3-486b-99cc-bf3f36a14347 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:49.582973 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:49.604721 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7049fa-6fc8-4351-9670-ab6fd597173f None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:49.661412 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3170499c-61e3-486b-99cc-bf3f36a14347 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:49.663424 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3170499c-61e3-486b-99cc-bf3f36a14347 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:49.667118 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651506e0-d024-4710-81cd-1f427b6e8f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:49.667760 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651506e0-d024-4710-81cd-1f427b6e8f1e None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:49.668037 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651506e0-d024-4710-81cd-1f427b6e8f1e None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:49.668311 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651506e0-d024-4710-81cd-1f427b6e8f1e None admin] PATH_INFO: `/v3/projects/7537ad5b7fef46308a69ce89f36209c2` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:49.670209 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651506e0-d024-4710-81cd-1f427b6e8f1e None admin] RBAC: Authorizing `identity:get_project(project_id=7537ad5b7fef46308a69ce89f36209c2)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:49.671652 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651506e0-d024-4710-81cd-1f427b6e8f1e None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:49.673307 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 92/182] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 08:31:49 2022] GET /identity/v3/projects/7537ad5b7fef46308a69ce89f36209c2 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 08:31:49.678931 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651506e0-d024-4710-81cd-1f427b6e8f1e None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:49.848357 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651506e0-d024-4710-81cd-1f427b6e8f1e None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:49.851954 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 91/183] 10.222.0.30 () {58 vars in 1282 bytes} [Sat Oct 15 08:31:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 609 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 08:31:49.859913 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3170499c-61e3-486b-99cc-bf3f36a14347 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:49.887685 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:49.924042 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2c2273-7769-4bb9-8b3f-27dc15f5a416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:49.924655 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c2273-7769-4bb9-8b3f-27dc15f5a416 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:49.924942 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c2273-7769-4bb9-8b3f-27dc15f5a416 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:49.925219 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c2273-7769-4bb9-8b3f-27dc15f5a416 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:49.925966 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2c2273-7769-4bb9-8b3f-27dc15f5a416 None admin] RBAC: Adding query filter params (role.id=14615e30ed3f4d8cb78ee5e0bf9b5761, scope.project.id=7537ad5b7fef46308a69ce89f36209c2, user.id=5ab448f9598c4699a79802391e4d7baa) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:31:49.926280 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2c2273-7769-4bb9-8b3f-27dc15f5a416 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:49.927787 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2c2273-7769-4bb9-8b3f-27dc15f5a416 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:49.939053 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 93/184] 10.222.0.30 () {56 vars in 1374 bytes} [Sat Oct 15 08:31:49 2022] GET /identity/v3/role_assignments?user.id=5ab448f9598c4699a79802391e4d7baa&scope.project.id=7537ad5b7fef46308a69ce89f36209c2&role.id=14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 608 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:50.011774 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3170499c-61e3-486b-99cc-bf3f36a14347 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:50.067455 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27b2b74-c543-4037-b749-d375763b3f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:50.068172 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b2b74-c543-4037-b749-d375763b3f73 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:50.068456 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b2b74-c543-4037-b749-d375763b3f73 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:50.068728 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b2b74-c543-4037-b749-d375763b3f73 None admin] PATH_INFO: `/v3/projects/7537ad5b7fef46308a69ce89f36209c2` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:50.069938 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27b2b74-c543-4037-b749-d375763b3f73 None admin] RBAC: Authorizing `identity:get_project(project_id=7537ad5b7fef46308a69ce89f36209c2)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:50.071532 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27b2b74-c543-4037-b749-d375763b3f73 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:50.074324 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 92/185] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 08:31:49 2022] GET /identity/v3/projects/7537ad5b7fef46308a69ce89f36209c2 => generated 302 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:50.079950 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2c2273-7769-4bb9-8b3f-27dc15f5a416 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:50.103304 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:50.201314 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2c2273-7769-4bb9-8b3f-27dc15f5a416 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:50.235234 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27771835-755e-4171-b89b-f21b8a809596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:50.235991 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27771835-755e-4171-b89b-f21b8a809596 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:50.236281 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27771835-755e-4171-b89b-f21b8a809596 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:50.236558 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27771835-755e-4171-b89b-f21b8a809596 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:50.237333 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27771835-755e-4171-b89b-f21b8a809596 None admin] RBAC: Adding query filter params (role.id=d236b691c1434103b73cc4fd0b39fec1, scope.project.id=7537ad5b7fef46308a69ce89f36209c2, user.id=5ab448f9598c4699a79802391e4d7baa) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:31:50.237646 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27771835-755e-4171-b89b-f21b8a809596 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:50.239104 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27771835-755e-4171-b89b-f21b8a809596 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:50.245927 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 94/186] 10.222.0.30 () {56 vars in 1374 bytes} [Sat Oct 15 08:31:50 2022] GET /identity/v3/role_assignments?user.id=5ab448f9598c4699a79802391e4d7baa&scope.project.id=7537ad5b7fef46308a69ce89f36209c2&role.id=d236b691c1434103b73cc4fd0b39fec1 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:50.691740 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f78be6-0c06-4264-8438-a058a291ff9c None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:50.692268 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f78be6-0c06-4264-8438-a058a291ff9c None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:50.692268 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f78be6-0c06-4264-8438-a058a291ff9c None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:50.694032 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 93/187] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:31:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:31:50.699643 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36208c82-8182-4696-94bc-3568f0b63e68 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:50.700057 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36208c82-8182-4696-94bc-3568f0b63e68 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:50.700246 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36208c82-8182-4696-94bc-3568f0b63e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:50.718033 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-36208c82-8182-4696-94bc-3568f0b63e68 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:31:50.719208 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-36208c82-8182-4696-94bc-3568f0b63e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:31:50.768595 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f7285f-ff47-4d28-9429-ea4c8e98740b None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:50.768887 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f7285f-ff47-4d28-9429-ea4c8e98740b None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:50.769170 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f7285f-ff47-4d28-9429-ea4c8e98740b None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:50.770689 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 94/188] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:31:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:31:50.775262 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93821be8-1a43-4248-9e9d-9447f51813f0 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:50.776107 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93821be8-1a43-4248-9e9d-9447f51813f0 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:50.776390 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93821be8-1a43-4248-9e9d-9447f51813f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:50.788733 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 95/189] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:31:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 08:31:50.794309 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-93821be8-1a43-4248-9e9d-9447f51813f0 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:31:50.795524 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-93821be8-1a43-4248-9e9d-9447f51813f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:31:50.862782 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 95/190] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:31:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 08:31:51.057582 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36208c82-8182-4696-94bc-3568f0b63e68 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:51.069142 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:51.129312 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93821be8-1a43-4248-9e9d-9447f51813f0 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:51.144722 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:51.162126 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36208c82-8182-4696-94bc-3568f0b63e68 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:51.200481 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad76fad-a4aa-47d0-81e9-4210a519c76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:51.201132 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad76fad-a4aa-47d0-81e9-4210a519c76f None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:51.201414 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad76fad-a4aa-47d0-81e9-4210a519c76f None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:51.201711 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad76fad-a4aa-47d0-81e9-4210a519c76f None admin] PATH_INFO: `/v3/domains/default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:51.202868 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad76fad-a4aa-47d0-81e9-4210a519c76f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:51.204326 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad76fad-a4aa-47d0-81e9-4210a519c76f None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:51.209708 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 96/191] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:31:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:51.211274 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad76fad-a4aa-47d0-81e9-4210a519c76f None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:51.236044 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93821be8-1a43-4248-9e9d-9447f51813f0 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:51.269993 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463345b6-3181-4b79-8edb-b19c4dc4b21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:51.270633 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463345b6-3181-4b79-8edb-b19c4dc4b21d None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:51.270916 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463345b6-3181-4b79-8edb-b19c4dc4b21d None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:51.271188 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463345b6-3181-4b79-8edb-b19c4dc4b21d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:51.282746 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463345b6-3181-4b79-8edb-b19c4dc4b21d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:31:51.287531 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-463345b6-3181-4b79-8edb-b19c4dc4b21d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 15 08:31:51.287531 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:31:51.287531 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:31:51.287531 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:31:51.287531 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:31:51.287531 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:31:51.287531 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:31:51.287531 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:31:51.287531 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:31:51.287531 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:31:51.287531 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:31:51.287531 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:31:51.287531 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:31:51.287531 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:31:51.287531 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:31:51.287531 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:31:51.288401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:31:51.288401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:31:51.288401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:31:51.288401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:31:51.288401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:31:51.288401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:31:51.288401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:31:51.288401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:31:51.288401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:31:51.288401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:31:51.288401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:31:51.288401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:31:51.288401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:31:51.288401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:31:51.288401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:31:51.288401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:31:51.289205 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:31:51.289205 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:31:51.289205 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:31:51.289205 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:31:51.289205 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:31:51.289205 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:31:51.289205 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:31:51.289205 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:31:51.289205 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:31:51.289205 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:31:51.289205 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 15 08:31:51.289205 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:31:51.289205 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 96/192] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 08:31:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 08:31:51.294501 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e8097-c6a6-44dd-8cbd-67c9999cbef2 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:51.294791 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e8097-c6a6-44dd-8cbd-67c9999cbef2 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:51.295073 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e8097-c6a6-44dd-8cbd-67c9999cbef2 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:51.296372 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 97/193] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:31:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:31:51.296935 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:51.298202 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7e8097-c6a6-44dd-8cbd-67c9999cbef2 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:51.319810 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad76fad-a4aa-47d0-81e9-4210a519c76f None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:51.352259 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7851a8-c5ef-4499-9aad-fbb2d086e759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:51.352857 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7851a8-c5ef-4499-9aad-fbb2d086e759 None admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:51.353140 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7851a8-c5ef-4499-9aad-fbb2d086e759 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:51.353416 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7851a8-c5ef-4499-9aad-fbb2d086e759 None admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:51.355409 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7851a8-c5ef-4499-9aad-fbb2d086e759 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:51.356954 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7851a8-c5ef-4499-9aad-fbb2d086e759 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:51.377832 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:51.378370 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 97/194] 10.222.0.30 () {60 vars in 1132 bytes} [Sat Oct 15 08:31:51 2022] POST /identity/v3/users => generated 293 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 08:31:51.379643 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cfa248-1b5f-4ba5-97e8-17b27a6e756e None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:51.379932 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cfa248-1b5f-4ba5-97e8-17b27a6e756e None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:51.380201 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cfa248-1b5f-4ba5-97e8-17b27a6e756e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:51.401626 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-b1cfa248-1b5f-4ba5-97e8-17b27a6e756e None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:31:51.402934 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-b1cfa248-1b5f-4ba5-97e8-17b27a6e756e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:31:51.419624 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7e8097-c6a6-44dd-8cbd-67c9999cbef2 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:51.457465 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c3fd89-6eeb-48df-8db3-da82e3f2e0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:51.458123 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c3fd89-6eeb-48df-8db3-da82e3f2e0dc None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:51.458404 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c3fd89-6eeb-48df-8db3-da82e3f2e0dc None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:51.458678 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c3fd89-6eeb-48df-8db3-da82e3f2e0dc None admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:51.459365 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c3fd89-6eeb-48df-8db3-da82e3f2e0dc None admin] RBAC: Adding query filter params (name=reader) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:31:51.459669 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c3fd89-6eeb-48df-8db3-da82e3f2e0dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:51.461084 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c3fd89-6eeb-48df-8db3-da82e3f2e0dc None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:51.470409 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 98/195] 10.222.0.30 () {56 vars in 1101 bytes} [Sat Oct 15 08:31:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:51.476270 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c3fd89-6eeb-48df-8db3-da82e3f2e0dc None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:51.486967 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 98/196] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:31:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 08:31:51.589029 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c3fd89-6eeb-48df-8db3-da82e3f2e0dc None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:51.624393 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3860d442-33e1-4807-b4f9-93e2555c6c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:51.625028 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3860d442-33e1-4807-b4f9-93e2555c6c47 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:51.625314 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3860d442-33e1-4807-b4f9-93e2555c6c47 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:51.625593 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3860d442-33e1-4807-b4f9-93e2555c6c47 None admin] PATH_INFO: `/v3/users/ed401930a06e457daeab938f2ddfede1` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:51.628314 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:51.640551 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3860d442-33e1-4807-b4f9-93e2555c6c47 None admin] RBAC: Authorizing `identity:get_user(user_id=ed401930a06e457daeab938f2ddfede1)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:51.642399 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3860d442-33e1-4807-b4f9-93e2555c6c47 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:51.644301 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 99/197] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 08:31:51 2022] GET /identity/v3/users/ed401930a06e457daeab938f2ddfede1 => generated 297 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 08:31:51.649783 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cfa248-1b5f-4ba5-97e8-17b27a6e756e None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:51.727494 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:51.757255 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3860d442-33e1-4807-b4f9-93e2555c6c47 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:51.777669 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cfa248-1b5f-4ba5-97e8-17b27a6e756e None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:51.815426 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8d552d-795b-4c9c-b69c-e72d11e72223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:51.816076 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8d552d-795b-4c9c-b69c-e72d11e72223 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:51.816361 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8d552d-795b-4c9c-b69c-e72d11e72223 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:51.816639 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8d552d-795b-4c9c-b69c-e72d11e72223 None admin] PATH_INFO: `/v3/projects/7537ad5b7fef46308a69ce89f36209c2` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:51.818018 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8d552d-795b-4c9c-b69c-e72d11e72223 None admin] RBAC: Authorizing `identity:get_project(project_id=7537ad5b7fef46308a69ce89f36209c2)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:51.819579 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8d552d-795b-4c9c-b69c-e72d11e72223 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:51.821329 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 99/198] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 08:31:51 2022] GET /identity/v3/projects/7537ad5b7fef46308a69ce89f36209c2 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:51.827601 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8d552d-795b-4c9c-b69c-e72d11e72223 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:51.886868 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3860d442-33e1-4807-b4f9-93e2555c6c47 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:51.933793 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241cd2e9-ae2b-4e86-af57-fd23872622aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:51.934697 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241cd2e9-ae2b-4e86-af57-fd23872622aa None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:51.934977 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241cd2e9-ae2b-4e86-af57-fd23872622aa None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:51.935247 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241cd2e9-ae2b-4e86-af57-fd23872622aa None admin] PATH_INFO: `/v3/roles/member` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:51.936364 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8d552d-795b-4c9c-b69c-e72d11e72223 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:51.948718 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241cd2e9-ae2b-4e86-af57-fd23872622aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:31:51.955507 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-241cd2e9-ae2b-4e86-af57-fd23872622aa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 15 08:31:51.955507 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:31:51.955507 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:31:51.955507 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:31:51.955507 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:31:51.955507 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:31:51.955507 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:31:51.955507 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:31:51.955507 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:31:51.955507 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:31:51.955507 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:31:51.955507 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:31:51.955507 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:31:51.955507 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:31:51.955507 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:31:51.955507 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:31:51.956876 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:31:51.956876 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:31:51.956876 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:31:51.956876 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:31:51.956876 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:31:51.956876 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:31:51.956876 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:31:51.956876 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:31:51.956876 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:31:51.956876 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:31:51.956876 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:31:51.956876 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:31:51.956876 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:31:51.956876 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:31:51.956876 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:31:51.956876 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:31:51.958686 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:31:51.958686 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:31:51.958686 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:31:51.958686 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:31:51.958686 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:31:51.958686 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:31:51.958686 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:31:51.958686 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:31:51.958686 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:31:51.958686 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:31:51.958686 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 15 08:31:51.958686 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:31:51.958686 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 100/199] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 08:31:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 08:31:51.965720 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241cd2e9-ae2b-4e86-af57-fd23872622aa None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:51.972910 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:51.978886 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37248bd0-d56b-4161-8212-67081f86979b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:51.980023 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37248bd0-d56b-4161-8212-67081f86979b None admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:51.980512 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37248bd0-d56b-4161-8212-67081f86979b None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:51.981091 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37248bd0-d56b-4161-8212-67081f86979b None admin] PATH_INFO: `/v3/projects/7537ad5b7fef46308a69ce89f36209c2/users/ed401930a06e457daeab938f2ddfede1/roles/080a66722a3d4936966a3a50bf9bafe1` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:51.983358 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37248bd0-d56b-4161-8212-67081f86979b None admin] RBAC: Authorizing `identity:create_grant(project_id=7537ad5b7fef46308a69ce89f36209c2, user_id=ed401930a06e457daeab938f2ddfede1, role_id=080a66722a3d4936966a3a50bf9bafe1)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:51.985203 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37248bd0-d56b-4161-8212-67081f86979b None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:52.005754 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 100/200] 10.222.0.30 () {58 vars in 1439 bytes} [Sat Oct 15 08:31:51 2022] PUT /identity/v3/projects/7537ad5b7fef46308a69ce89f36209c2/users/ed401930a06e457daeab938f2ddfede1/roles/080a66722a3d4936966a3a50bf9bafe1 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 08:31:52.042835 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:52.094409 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241cd2e9-ae2b-4e86-af57-fd23872622aa None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:52.133325 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63252587-ddba-482f-b2c3-6dfc2089f119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:52.134223 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63252587-ddba-482f-b2c3-6dfc2089f119 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:52.135265 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63252587-ddba-482f-b2c3-6dfc2089f119 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:52.135767 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63252587-ddba-482f-b2c3-6dfc2089f119 None admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:52.136588 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63252587-ddba-482f-b2c3-6dfc2089f119 None admin] RBAC: Adding query filter params (name=member) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:31:52.137106 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63252587-ddba-482f-b2c3-6dfc2089f119 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:52.138733 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63252587-ddba-482f-b2c3-6dfc2089f119 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:52.148929 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 101/201] 10.222.0.30 () {56 vars in 1101 bytes} [Sat Oct 15 08:31:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:52.155236 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37248bd0-d56b-4161-8212-67081f86979b None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:52.262919 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37248bd0-d56b-4161-8212-67081f86979b None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:52.298903 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976d2627-4b5e-4961-a51b-a72c73b346f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:52.299782 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976d2627-4b5e-4961-a51b-a72c73b346f9 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:52.300266 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976d2627-4b5e-4961-a51b-a72c73b346f9 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:52.300681 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976d2627-4b5e-4961-a51b-a72c73b346f9 None admin] PATH_INFO: `/v3/users/299f17920261414a905a73845424d7fa` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:52.301995 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976d2627-4b5e-4961-a51b-a72c73b346f9 None admin] RBAC: Authorizing `identity:get_user(user_id=299f17920261414a905a73845424d7fa)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:52.303560 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976d2627-4b5e-4961-a51b-a72c73b346f9 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:52.305567 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 101/202] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 08:31:52 2022] GET /identity/v3/users/299f17920261414a905a73845424d7fa => generated 297 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:52.311345 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63252587-ddba-482f-b2c3-6dfc2089f119 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:52.416926 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63252587-ddba-482f-b2c3-6dfc2089f119 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:52.434901 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:52.453539 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b798d1-a926-4e53-9521-3be6ae2efa96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:52.454179 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b798d1-a926-4e53-9521-3be6ae2efa96 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:52.454458 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b798d1-a926-4e53-9521-3be6ae2efa96 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:52.454732 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b798d1-a926-4e53-9521-3be6ae2efa96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:52.538559 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b798d1-a926-4e53-9521-3be6ae2efa96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:52.540041 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b798d1-a926-4e53-9521-3be6ae2efa96 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:52.638988 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 102/203] 10.222.0.30 () {58 vars in 1282 bytes} [Sat Oct 15 08:31:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 08:31:52.644765 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976d2627-4b5e-4961-a51b-a72c73b346f9 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:52.706687 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:52.749726 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976d2627-4b5e-4961-a51b-a72c73b346f9 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:52.784257 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24381486-0153-42b4-bfa2-a644e98bd5b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:52.784894 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24381486-0153-42b4-bfa2-a644e98bd5b4 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:52.785171 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24381486-0153-42b4-bfa2-a644e98bd5b4 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:52.785444 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24381486-0153-42b4-bfa2-a644e98bd5b4 None admin] PATH_INFO: `/v3/projects/7537ad5b7fef46308a69ce89f36209c2` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:52.786599 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24381486-0153-42b4-bfa2-a644e98bd5b4 None admin] RBAC: Authorizing `identity:get_project(project_id=7537ad5b7fef46308a69ce89f36209c2)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:52.788048 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24381486-0153-42b4-bfa2-a644e98bd5b4 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:52.789766 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 102/204] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 08:31:52 2022] GET /identity/v3/projects/7537ad5b7fef46308a69ce89f36209c2 => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:52.795677 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b798d1-a926-4e53-9521-3be6ae2efa96 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:52.814393 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:52.897672 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b798d1-a926-4e53-9521-3be6ae2efa96 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:52.932974 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd248981-b593-4412-b47c-66f46e3c55f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:52.933581 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd248981-b593-4412-b47c-66f46e3c55f9 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:52.933885 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd248981-b593-4412-b47c-66f46e3c55f9 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:52.934161 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd248981-b593-4412-b47c-66f46e3c55f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:52.934877 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd248981-b593-4412-b47c-66f46e3c55f9 None admin] RBAC: Adding query filter params (role.id=3248f0268738489b883041851d681bb7, scope.project.id=7537ad5b7fef46308a69ce89f36209c2, user.id=299f17920261414a905a73845424d7fa) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:31:52.935183 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd248981-b593-4412-b47c-66f46e3c55f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:52.936623 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd248981-b593-4412-b47c-66f46e3c55f9 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:52.942366 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 103/205] 10.222.0.30 () {56 vars in 1374 bytes} [Sat Oct 15 08:31:52 2022] GET /identity/v3/role_assignments?user.id=299f17920261414a905a73845424d7fa&scope.project.id=7537ad5b7fef46308a69ce89f36209c2&role.id=3248f0268738489b883041851d681bb7 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:53.697826 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354fe9f5-38bb-4c5a-b1f6-692f0c15dd3d None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:53.698189 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354fe9f5-38bb-4c5a-b1f6-692f0c15dd3d None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:53.698369 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354fe9f5-38bb-4c5a-b1f6-692f0c15dd3d None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:53.699810 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 103/206] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:31:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:31:53.704843 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378a9a9-c7e5-44d2-96e3-3cd931a0ae28 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:53.705145 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378a9a9-c7e5-44d2-96e3-3cd931a0ae28 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:53.705428 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378a9a9-c7e5-44d2-96e3-3cd931a0ae28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:53.723739 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-4378a9a9-c7e5-44d2-96e3-3cd931a0ae28 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:31:53.724948 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-4378a9a9-c7e5-44d2-96e3-3cd931a0ae28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:31:53.745967 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361d12d8-69db-400d-9a3e-d6ce49fa15fc None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:53.746260 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361d12d8-69db-400d-9a3e-d6ce49fa15fc None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:53.746544 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361d12d8-69db-400d-9a3e-d6ce49fa15fc None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:53.747910 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 104/207] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:31:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 08:31:53.752724 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceee126-c7f3-4d25-b9b3-785b2a2e5b6c None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:53.753290 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceee126-c7f3-4d25-b9b3-785b2a2e5b6c None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:53.753290 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceee126-c7f3-4d25-b9b3-785b2a2e5b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:53.772653 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-eceee126-c7f3-4d25-b9b3-785b2a2e5b6c None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:31:53.774124 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-eceee126-c7f3-4d25-b9b3-785b2a2e5b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:31:53.798220 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 104/208] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:31:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 08:31:53.842801 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 105/209] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:31:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 08:31:54.073534 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4378a9a9-c7e5-44d2-96e3-3cd931a0ae28 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:54.085174 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:54.106463 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceee126-c7f3-4d25-b9b3-785b2a2e5b6c None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:54.169944 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:54.185609 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4378a9a9-c7e5-44d2-96e3-3cd931a0ae28 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:54.215717 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceee126-c7f3-4d25-b9b3-785b2a2e5b6c None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:54.220765 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a069e571-c05b-4a86-9f8b-ce46ab76d28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:54.221394 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a069e571-c05b-4a86-9f8b-ce46ab76d28c None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:54.221671 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a069e571-c05b-4a86-9f8b-ce46ab76d28c None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:54.221967 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a069e571-c05b-4a86-9f8b-ce46ab76d28c None admin] PATH_INFO: `/v3/roles/member` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:54.232387 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a069e571-c05b-4a86-9f8b-ce46ab76d28c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:31:54.237248 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-a069e571-c05b-4a86-9f8b-ce46ab76d28c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 15 08:31:54.237248 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:31:54.237248 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:31:54.237248 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:31:54.237248 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:31:54.237248 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:31:54.237248 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:31:54.237248 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:31:54.237248 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:31:54.237248 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:31:54.237248 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:31:54.237248 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:31:54.237248 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:31:54.237248 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:31:54.237248 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:31:54.237248 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:31:54.238170 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:31:54.238170 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:31:54.238170 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:31:54.238170 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:31:54.238170 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:31:54.238170 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:31:54.238170 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:31:54.238170 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:31:54.238170 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:31:54.238170 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:31:54.238170 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:31:54.238170 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:31:54.238170 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:31:54.238170 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:31:54.238170 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:31:54.238170 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:31:54.238972 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:31:54.238972 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:31:54.238972 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:31:54.238972 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:31:54.238972 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:31:54.238972 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:31:54.238972 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:31:54.238972 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:31:54.238972 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:31:54.238972 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:31:54.238972 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 15 08:31:54.238972 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:31:54.238972 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 105/210] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 08:31:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:31:54.243949 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a069e571-c05b-4a86-9f8b-ce46ab76d28c None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:54.257136 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0ed3b5-133d-4fed-be35-6ded063cb39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:54.257820 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0ed3b5-133d-4fed-be35-6ded063cb39b None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:54.258112 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0ed3b5-133d-4fed-be35-6ded063cb39b None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:54.258393 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0ed3b5-133d-4fed-be35-6ded063cb39b None admin] PATH_INFO: `/v3/domains/default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:54.259502 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0ed3b5-133d-4fed-be35-6ded063cb39b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:54.261010 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0ed3b5-133d-4fed-be35-6ded063cb39b None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:54.262625 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 106/211] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:31:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 08:31:54.263754 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d69256-1c0b-4a65-b393-9b3f2aa0dec1 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:54.264041 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d69256-1c0b-4a65-b393-9b3f2aa0dec1 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:54.264318 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d69256-1c0b-4a65-b393-9b3f2aa0dec1 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:54.265518 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 107/212] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:31:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:31:54.268904 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d69256-1c0b-4a65-b393-9b3f2aa0dec1 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:54.318573 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:54.348931 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a069e571-c05b-4a86-9f8b-ce46ab76d28c None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:54.370610 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d69256-1c0b-4a65-b393-9b3f2aa0dec1 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:54.382890 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5470000c-1a27-48df-b65e-2106ef4d3bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:54.383496 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5470000c-1a27-48df-b65e-2106ef4d3bef None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:54.383775 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5470000c-1a27-48df-b65e-2106ef4d3bef None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:54.384046 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5470000c-1a27-48df-b65e-2106ef4d3bef None admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:54.384704 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5470000c-1a27-48df-b65e-2106ef4d3bef None admin] RBAC: Adding query filter params (name=member) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:31:54.385005 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5470000c-1a27-48df-b65e-2106ef4d3bef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:54.386440 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5470000c-1a27-48df-b65e-2106ef4d3bef None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:54.394872 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 106/213] 10.222.0.30 () {56 vars in 1101 bytes} [Sat Oct 15 08:31:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 08:31:54.396032 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb0e4d-7f48-4280-885a-9bfe7429aae7 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:54.396314 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb0e4d-7f48-4280-885a-9bfe7429aae7 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:54.396588 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb0e4d-7f48-4280-885a-9bfe7429aae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:54.401119 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:54.411617 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c42650-b158-4cac-a5f0-e1fdb1372d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:54.412229 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c42650-b158-4cac-a5f0-e1fdb1372d0b None admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:54.412514 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c42650-b158-4cac-a5f0-e1fdb1372d0b None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:54.412793 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c42650-b158-4cac-a5f0-e1fdb1372d0b None admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:54.413443 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c42650-b158-4cac-a5f0-e1fdb1372d0b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:54.414414 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-c7bb0e4d-7f48-4280-885a-9bfe7429aae7 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:31:54.415208 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c42650-b158-4cac-a5f0-e1fdb1372d0b None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:54.415607 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-c7bb0e4d-7f48-4280-885a-9bfe7429aae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:31:54.436737 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 108/214] 10.222.0.30 () {60 vars in 1132 bytes} [Sat Oct 15 08:31:54 2022] POST /identity/v3/users => generated 293 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 08:31:54.437564 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c42650-b158-4cac-a5f0-e1fdb1372d0b None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:54.510326 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 107/215] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:31:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 08:31:54.551179 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c42650-b158-4cac-a5f0-e1fdb1372d0b None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:54.587330 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183fb5a0-56ab-4cc6-8631-f4e83437bcbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:54.588549 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183fb5a0-56ab-4cc6-8631-f4e83437bcbe None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:54.588832 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183fb5a0-56ab-4cc6-8631-f4e83437bcbe None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:54.589105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183fb5a0-56ab-4cc6-8631-f4e83437bcbe None admin] PATH_INFO: `/v3/users/ebfaacb4deeb4a208cfaf1b6f933b2f4` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:54.603655 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183fb5a0-56ab-4cc6-8631-f4e83437bcbe None admin] RBAC: Authorizing `identity:get_user(user_id=ebfaacb4deeb4a208cfaf1b6f933b2f4)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:54.605051 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183fb5a0-56ab-4cc6-8631-f4e83437bcbe None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:54.606943 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 109/216] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 08:31:54 2022] GET /identity/v3/users/ebfaacb4deeb4a208cfaf1b6f933b2f4 => generated 293 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:54.613075 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bb0e4d-7f48-4280-885a-9bfe7429aae7 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:54.685527 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:54.728287 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bb0e4d-7f48-4280-885a-9bfe7429aae7 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:54.767170 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f33f7e5-5ca9-4227-939a-8513faf95159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:54.768039 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f33f7e5-5ca9-4227-939a-8513faf95159 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:54.768475 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f33f7e5-5ca9-4227-939a-8513faf95159 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:54.768896 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f33f7e5-5ca9-4227-939a-8513faf95159 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:54.769844 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f33f7e5-5ca9-4227-939a-8513faf95159 None admin] RBAC: Adding query filter params (role.id=3248f0268738489b883041851d681bb7, user.id=ebfaacb4deeb4a208cfaf1b6f933b2f4, scope.system=all) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:31:54.770312 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f33f7e5-5ca9-4227-939a-8513faf95159 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:54.771782 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f33f7e5-5ca9-4227-939a-8513faf95159 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:54.777323 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 108/217] 10.222.0.30 () {56 vars in 1308 bytes} [Sat Oct 15 08:31:54 2022] GET /identity/v3/role_assignments?user.id=ebfaacb4deeb4a208cfaf1b6f933b2f4&scope.system=all&role.id=3248f0268738489b883041851d681bb7 => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:54.778427 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183fb5a0-56ab-4cc6-8631-f4e83437bcbe None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:54.833501 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:54.899521 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183fb5a0-56ab-4cc6-8631-f4e83437bcbe None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:54.938794 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382a6927-ab08-4340-b26e-afe7e31606a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:54.939459 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a6927-ab08-4340-b26e-afe7e31606a3 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:54.939743 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a6927-ab08-4340-b26e-afe7e31606a3 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:54.940021 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a6927-ab08-4340-b26e-afe7e31606a3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:54.952575 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382a6927-ab08-4340-b26e-afe7e31606a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:31:54.957472 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-382a6927-ab08-4340-b26e-afe7e31606a3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 15 08:31:54.957472 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:31:54.957472 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:31:54.957472 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:31:54.957472 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:31:54.957472 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:31:54.957472 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:31:54.957472 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:31:54.957472 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:31:54.957472 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:31:54.957472 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:31:54.957472 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:31:54.957472 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:31:54.957472 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:31:54.957472 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:31:54.957472 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:31:54.958385 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:31:54.958385 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:31:54.958385 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:31:54.958385 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:31:54.958385 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:31:54.958385 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:31:54.958385 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:31:54.958385 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:31:54.958385 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:31:54.958385 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:31:54.958385 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:31:54.958385 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:31:54.958385 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:31:54.958385 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:31:54.958385 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:31:54.958385 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:31:54.959608 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:31:54.959608 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:31:54.959608 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:31:54.959608 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:31:54.959608 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:31:54.959608 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:31:54.959608 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:31:54.959608 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:31:54.959608 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:31:54.959608 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:31:54.959608 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 15 08:31:54.959608 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:31:54.959608 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 110/218] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 08:31:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 08:31:54.965291 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f33f7e5-5ca9-4227-939a-8513faf95159 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:55.070650 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f33f7e5-5ca9-4227-939a-8513faf95159 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:55.098427 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:55.108196 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f1ec0f-a5f6-473f-85ca-12a63ae9f6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:55.108817 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f1ec0f-a5f6-473f-85ca-12a63ae9f6aa None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:55.109099 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f1ec0f-a5f6-473f-85ca-12a63ae9f6aa None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:55.109377 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f1ec0f-a5f6-473f-85ca-12a63ae9f6aa None admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:55.110116 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f1ec0f-a5f6-473f-85ca-12a63ae9f6aa None admin] RBAC: Adding query filter params (name=reader) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:31:55.110421 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f1ec0f-a5f6-473f-85ca-12a63ae9f6aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:55.111806 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f1ec0f-a5f6-473f-85ca-12a63ae9f6aa None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:55.121230 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 109/219] 10.222.0.30 () {56 vars in 1101 bytes} [Sat Oct 15 08:31:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:55.126892 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382a6927-ab08-4340-b26e-afe7e31606a3 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:55.231042 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:55.234074 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382a6927-ab08-4340-b26e-afe7e31606a3 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:55.270209 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9042117-0740-4cca-a853-24b4c067d36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:55.270854 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9042117-0740-4cca-a853-24b4c067d36f None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:55.271137 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9042117-0740-4cca-a853-24b4c067d36f None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:55.271472 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9042117-0740-4cca-a853-24b4c067d36f None admin] PATH_INFO: `/v3/users/ed401930a06e457daeab938f2ddfede1` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:55.272613 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9042117-0740-4cca-a853-24b4c067d36f None admin] RBAC: Authorizing `identity:get_user(user_id=ed401930a06e457daeab938f2ddfede1)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:55.274104 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9042117-0740-4cca-a853-24b4c067d36f None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:55.275755 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 111/220] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 08:31:55 2022] GET /identity/v3/users/ed401930a06e457daeab938f2ddfede1 => generated 297 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:55.281306 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f1ec0f-a5f6-473f-85ca-12a63ae9f6aa None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:55.385164 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f1ec0f-a5f6-473f-85ca-12a63ae9f6aa None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:55.424730 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0627feca-0af4-4770-a349-8bc75648bab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:55.425354 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0627feca-0af4-4770-a349-8bc75648bab3 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:55.425639 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0627feca-0af4-4770-a349-8bc75648bab3 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:55.425946 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0627feca-0af4-4770-a349-8bc75648bab3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:55.488558 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:55.501227 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0627feca-0af4-4770-a349-8bc75648bab3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:55.503251 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0627feca-0af4-4770-a349-8bc75648bab3 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:55.609377 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 110/221] 10.222.0.30 () {58 vars in 1282 bytes} [Sat Oct 15 08:31:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 08:31:55.617709 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9042117-0740-4cca-a853-24b4c067d36f None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:55.724197 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9042117-0740-4cca-a853-24b4c067d36f None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:55.758820 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a428252c-ce99-49ab-b458-20263bf8d605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:55.759463 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a428252c-ce99-49ab-b458-20263bf8d605 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:55.759793 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a428252c-ce99-49ab-b458-20263bf8d605 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:55.760075 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a428252c-ce99-49ab-b458-20263bf8d605 None admin] PATH_INFO: `/v3/projects/7537ad5b7fef46308a69ce89f36209c2` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:55.761227 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a428252c-ce99-49ab-b458-20263bf8d605 None admin] RBAC: Authorizing `identity:get_project(project_id=7537ad5b7fef46308a69ce89f36209c2)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:55.762720 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a428252c-ce99-49ab-b458-20263bf8d605 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:55.764607 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 112/222] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 08:31:55 2022] GET /identity/v3/projects/7537ad5b7fef46308a69ce89f36209c2 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:55.770701 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0627feca-0af4-4770-a349-8bc75648bab3 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:55.860677 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:55.873328 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0627feca-0af4-4770-a349-8bc75648bab3 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:55.908010 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6153aa-e51e-4fdc-9902-fe8e20e7b712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:55.908616 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6153aa-e51e-4fdc-9902-fe8e20e7b712 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:55.908892 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6153aa-e51e-4fdc-9902-fe8e20e7b712 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:55.909162 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6153aa-e51e-4fdc-9902-fe8e20e7b712 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:55.909924 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6153aa-e51e-4fdc-9902-fe8e20e7b712 None admin] RBAC: Adding query filter params (role.id=080a66722a3d4936966a3a50bf9bafe1, scope.project.id=7537ad5b7fef46308a69ce89f36209c2, user.id=ed401930a06e457daeab938f2ddfede1) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:31:55.910227 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6153aa-e51e-4fdc-9902-fe8e20e7b712 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:55.911664 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6153aa-e51e-4fdc-9902-fe8e20e7b712 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:55.918500 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 111/223] 10.222.0.30 () {56 vars in 1374 bytes} [Sat Oct 15 08:31:55 2022] GET /identity/v3/role_assignments?user.id=ed401930a06e457daeab938f2ddfede1&scope.project.id=7537ad5b7fef46308a69ce89f36209c2&role.id=080a66722a3d4936966a3a50bf9bafe1 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:56.842138 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3726c9-c9cb-4b20-8bfc-757b0636c91d None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:56.842682 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3726c9-c9cb-4b20-8bfc-757b0636c91d None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:56.842682 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3726c9-c9cb-4b20-8bfc-757b0636c91d None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:56.844077 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 113/224] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:31:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:31:56.847159 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c389481-bb1c-463f-b9b0-c438ec71c57b None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:56.847573 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c389481-bb1c-463f-b9b0-c438ec71c57b None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:56.847712 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c389481-bb1c-463f-b9b0-c438ec71c57b None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:56.849241 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a93e2-7fd5-413b-9d9f-e239cb7ee6be None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:56.849666 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a93e2-7fd5-413b-9d9f-e239cb7ee6be None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:56.849791 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 112/225] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:31:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:31:56.849863 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a93e2-7fd5-413b-9d9f-e239cb7ee6be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:56.854603 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a83ae8-a3da-4787-91bb-3e1a37439dd1 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:56.854889 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a83ae8-a3da-4787-91bb-3e1a37439dd1 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:56.855166 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a83ae8-a3da-4787-91bb-3e1a37439dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:56.868661 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-677a93e2-7fd5-413b-9d9f-e239cb7ee6be None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:31:56.869917 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-677a93e2-7fd5-413b-9d9f-e239cb7ee6be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:31:56.874085 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-a7a83ae8-a3da-4787-91bb-3e1a37439dd1 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:31:56.875252 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-a7a83ae8-a3da-4787-91bb-3e1a37439dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:31:56.902764 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:56.944389 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 114/226] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:31:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 08:31:56.953002 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 113/227] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:31:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 08:31:57.004079 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589987ae-c3b1-4873-90ee-ce0a69092caa None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:57.004362 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589987ae-c3b1-4873-90ee-ce0a69092caa None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:57.004798 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589987ae-c3b1-4873-90ee-ce0a69092caa None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:57.006301 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 115/228] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:31:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:31:57.011947 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1f2052-444b-4e54-94d4-fa3080cd0535 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:57.012382 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1f2052-444b-4e54-94d4-fa3080cd0535 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:57.012798 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1f2052-444b-4e54-94d4-fa3080cd0535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:57.030884 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-2f1f2052-444b-4e54-94d4-fa3080cd0535 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:31:57.032219 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-2f1f2052-444b-4e54-94d4-fa3080cd0535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:31:57.098812 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 114/229] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:31:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 08:31:57.216106 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589987ae-c3b1-4873-90ee-ce0a69092caa None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:57.219366 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1f2052-444b-4e54-94d4-fa3080cd0535 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:57.234149 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:57.328946 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1f2052-444b-4e54-94d4-fa3080cd0535 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:57.329295 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589987ae-c3b1-4873-90ee-ce0a69092caa None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:57.364569 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adac759a-f4f2-4c49-bdc5-4741da3fb549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:57.365126 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac759a-f4f2-4c49-bdc5-4741da3fb549 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:57.365405 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac759a-f4f2-4c49-bdc5-4741da3fb549 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:57.365692 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac759a-f4f2-4c49-bdc5-4741da3fb549 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:57.366784 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adac759a-f4f2-4c49-bdc5-4741da3fb549 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:57.368240 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adac759a-f4f2-4c49-bdc5-4741da3fb549 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:57.369901 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 115/230] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:31:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:57.370147 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d5f33f-5e59-4172-9a4d-a0e93300b002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:57.370998 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d5f33f-5e59-4172-9a4d-a0e93300b002 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:57.371421 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d5f33f-5e59-4172-9a4d-a0e93300b002 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:57.371839 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d5f33f-5e59-4172-9a4d-a0e93300b002 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:57.375206 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adac759a-f4f2-4c49-bdc5-4741da3fb549 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:57.382276 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d5f33f-5e59-4172-9a4d-a0e93300b002 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:31:57.387770 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-70d5f33f-5e59-4172-9a4d-a0e93300b002 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 15 08:31:57.387770 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:31:57.387770 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:31:57.387770 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:31:57.387770 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:31:57.387770 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:31:57.387770 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:31:57.387770 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:31:57.387770 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:31:57.387770 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:31:57.387770 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:31:57.387770 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:31:57.387770 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:31:57.387770 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:31:57.387770 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:31:57.387770 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:31:57.388708 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:31:57.388708 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:31:57.388708 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:31:57.388708 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:31:57.388708 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:31:57.388708 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:31:57.388708 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:31:57.388708 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:31:57.388708 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:31:57.388708 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:31:57.388708 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:31:57.388708 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:31:57.388708 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:31:57.388708 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:31:57.388708 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:31:57.388708 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:31:57.389534 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:31:57.389534 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:31:57.389534 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:31:57.389534 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:31:57.389534 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:31:57.389534 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:31:57.389534 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:31:57.389534 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:31:57.389534 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:31:57.389534 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:31:57.389534 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 15 08:31:57.389534 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:31:57.389534 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 116/231] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 08:31:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:31:57.390224 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d5f33f-5e59-4172-9a4d-a0e93300b002 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:57.419837 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:57.459413 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:57.478778 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adac759a-f4f2-4c49-bdc5-4741da3fb549 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:57.491959 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d5f33f-5e59-4172-9a4d-a0e93300b002 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:57.512557 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3c9020-dc71-4af9-8635-708668e0678d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:57.513189 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3c9020-dc71-4af9-8635-708668e0678d None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:57.513471 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3c9020-dc71-4af9-8635-708668e0678d None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:57.513778 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3c9020-dc71-4af9-8635-708668e0678d None admin] PATH_INFO: `/v3/roles/member` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:57.523967 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3c9020-dc71-4af9-8635-708668e0678d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:31:57.524886 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c021d-707c-45bf-a2bf-f9839e417e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:57.525531 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c021d-707c-45bf-a2bf-f9839e417e2a None admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:57.525844 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c021d-707c-45bf-a2bf-f9839e417e2a None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:57.526115 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c021d-707c-45bf-a2bf-f9839e417e2a None admin] PATH_INFO: `/v3/groups` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:57.526735 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33c021d-707c-45bf-a2bf-f9839e417e2a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:57.528154 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33c021d-707c-45bf-a2bf-f9839e417e2a None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:57.528862 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-1d3c9020-dc71-4af9-8635-708668e0678d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 15 08:31:57.528862 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:31:57.528862 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:31:57.528862 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:31:57.528862 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:31:57.528862 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:31:57.528862 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:31:57.528862 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:31:57.528862 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:31:57.528862 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:31:57.528862 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:31:57.528862 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:31:57.528862 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:31:57.528862 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:31:57.528862 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:31:57.528862 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:31:57.529892 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:31:57.529892 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:31:57.529892 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:31:57.529892 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:31:57.529892 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:31:57.529892 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:31:57.529892 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:31:57.529892 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:31:57.529892 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:31:57.529892 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:31:57.529892 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:31:57.529892 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:31:57.529892 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:31:57.529892 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:31:57.529892 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:31:57.529892 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:31:57.531173 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:31:57.531173 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:31:57.531173 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:31:57.531173 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:31:57.531173 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:31:57.531173 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:31:57.531173 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:31:57.531173 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:31:57.531173 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:31:57.531173 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:31:57.531173 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 15 08:31:57.531173 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:31:57.531173 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 116/232] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 08:31:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:31:57.531173 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3c9020-dc71-4af9-8635-708668e0678d None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:57.545250 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 117/233] 10.222.0.30 () {60 vars in 1134 bytes} [Sat Oct 15 08:31:57 2022] POST /identity/v3/groups => generated 229 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:31:57.546098 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c021d-707c-45bf-a2bf-f9839e417e2a None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:57.644435 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3c9020-dc71-4af9-8635-708668e0678d None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:57.646315 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:57.669628 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c021d-707c-45bf-a2bf-f9839e417e2a None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:57.683006 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e473655-38be-4cd7-9eeb-9e960feedea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:57.683684 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e473655-38be-4cd7-9eeb-9e960feedea9 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:57.683970 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e473655-38be-4cd7-9eeb-9e960feedea9 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:57.684241 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e473655-38be-4cd7-9eeb-9e960feedea9 None admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:57.684916 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e473655-38be-4cd7-9eeb-9e960feedea9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:31:57.685220 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e473655-38be-4cd7-9eeb-9e960feedea9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:57.686676 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e473655-38be-4cd7-9eeb-9e960feedea9 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:57.696174 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 117/234] 10.222.0.30 () {56 vars in 1101 bytes} [Sat Oct 15 08:31:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:57.702700 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e473655-38be-4cd7-9eeb-9e960feedea9 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:57.705911 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:57.706752 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a68e99-bd0e-492d-a033-d0dee2f51c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:57.707452 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a68e99-bd0e-492d-a033-d0dee2f51c88 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:57.707827 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a68e99-bd0e-492d-a033-d0dee2f51c88 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:57.708059 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a68e99-bd0e-492d-a033-d0dee2f51c88 None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:57.708757 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a68e99-bd0e-492d-a033-d0dee2f51c88 None admin] RBAC: Adding query filter params (name=member) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:31:57.709096 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a68e99-bd0e-492d-a033-d0dee2f51c88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:57.710512 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a68e99-bd0e-492d-a033-d0dee2f51c88 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:57.721885 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 118/235] 10.222.0.30 () {56 vars in 1101 bytes} [Sat Oct 15 08:31:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:57.726727 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a68e99-bd0e-492d-a033-d0dee2f51c88 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:57.827205 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e473655-38be-4cd7-9eeb-9e960feedea9 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:57.831616 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a68e99-bd0e-492d-a033-d0dee2f51c88 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:57.863664 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289b897f-c746-49af-87c2-eabb5887285e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:57.864295 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289b897f-c746-49af-87c2-eabb5887285e None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:57.864570 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289b897f-c746-49af-87c2-eabb5887285e None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:57.864843 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289b897f-c746-49af-87c2-eabb5887285e None admin] PATH_INFO: `/v3/users/8afe315855ea4c18ab8570ce4f861acd` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:57.866672 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b0c85a-6321-4b18-8c4d-da08f43da396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:57.867307 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b0c85a-6321-4b18-8c4d-da08f43da396 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:57.867591 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b0c85a-6321-4b18-8c4d-da08f43da396 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:57.867875 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b0c85a-6321-4b18-8c4d-da08f43da396 None admin] PATH_INFO: `/v3/users/ebfaacb4deeb4a208cfaf1b6f933b2f4` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:57.869541 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b0c85a-6321-4b18-8c4d-da08f43da396 None admin] RBAC: Authorizing `identity:get_user(user_id=ebfaacb4deeb4a208cfaf1b6f933b2f4)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:57.871023 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b0c85a-6321-4b18-8c4d-da08f43da396 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:57.872764 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 119/236] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 08:31:57 2022] GET /identity/v3/users/ebfaacb4deeb4a208cfaf1b6f933b2f4 => generated 293 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 08:31:57.878040 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b0c85a-6321-4b18-8c4d-da08f43da396 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:57.880548 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289b897f-c746-49af-87c2-eabb5887285e None admin] RBAC: Authorizing `identity:get_user(user_id=8afe315855ea4c18ab8570ce4f861acd)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:57.882312 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289b897f-c746-49af-87c2-eabb5887285e None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:57.888729 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 118/237] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 08:31:57 2022] GET /identity/v3/users/8afe315855ea4c18ab8570ce4f861acd => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 08:31:57.889782 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289b897f-c746-49af-87c2-eabb5887285e None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:31:57.899034 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:57.949648 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:31:57.991402 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b0c85a-6321-4b18-8c4d-da08f43da396 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:57.992378 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289b897f-c746-49af-87c2-eabb5887285e None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:31:58.028508 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79974cd1-82e8-4daf-9841-0f3bc31b376a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:58.029115 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79974cd1-82e8-4daf-9841-0f3bc31b376a None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:58.029398 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79974cd1-82e8-4daf-9841-0f3bc31b376a None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:58.029495 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72593198-529c-40b7-8927-348abd0842a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:31:58.029713 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79974cd1-82e8-4daf-9841-0f3bc31b376a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:58.030129 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72593198-529c-40b7-8927-348abd0842a4 None admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:58.030414 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72593198-529c-40b7-8927-348abd0842a4 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:58.030503 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79974cd1-82e8-4daf-9841-0f3bc31b376a None admin] RBAC: Adding query filter params (role.id=080a66722a3d4936966a3a50bf9bafe1, user.id=8afe315855ea4c18ab8570ce4f861acd, scope.system=all) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:31:58.030694 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72593198-529c-40b7-8927-348abd0842a4 None admin] PATH_INFO: `/v3/system/users/ebfaacb4deeb4a208cfaf1b6f933b2f4/roles/3248f0268738489b883041851d681bb7` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:58.030787 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79974cd1-82e8-4daf-9841-0f3bc31b376a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:58.032178 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79974cd1-82e8-4daf-9841-0f3bc31b376a None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:58.032443 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72593198-529c-40b7-8927-348abd0842a4 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=ebfaacb4deeb4a208cfaf1b6f933b2f4, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:31:58.034084 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72593198-529c-40b7-8927-348abd0842a4 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:31:58.038120 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 119/238] 10.222.0.30 () {56 vars in 1308 bytes} [Sat Oct 15 08:31:57 2022] GET /identity/v3/role_assignments?user.id=8afe315855ea4c18ab8570ce4f861acd&scope.system=all&role.id=080a66722a3d4936966a3a50bf9bafe1 => generated 226 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:31:58.044372 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 120/239] 10.222.0.30 () {58 vars in 1334 bytes} [Sat Oct 15 08:31:57 2022] PUT /identity/v3/system/users/ebfaacb4deeb4a208cfaf1b6f933b2f4/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 08:31:59.740408 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732986b5-b95b-4e65-874a-448e774f4522 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:59.740891 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732986b5-b95b-4e65-874a-448e774f4522 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:59.740891 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732986b5-b95b-4e65-874a-448e774f4522 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:59.742288 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 120/240] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:31:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:31:59.747555 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b6c70-1aa3-4f35-bbdb-5458636259c4 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:31:59.747967 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b6c70-1aa3-4f35-bbdb-5458636259c4 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:31:59.748135 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b6c70-1aa3-4f35-bbdb-5458636259c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:31:59.767001 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-505b6c70-1aa3-4f35-bbdb-5458636259c4 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:31:59.768184 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-505b6c70-1aa3-4f35-bbdb-5458636259c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:31:59.834107 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 121/241] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:31:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 08:32:00.102658 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732986b5-b95b-4e65-874a-448e774f4522 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:00.190681 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c12339f-7b1c-4085-83bb-a362a34f659e None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:00.191062 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c12339f-7b1c-4085-83bb-a362a34f659e None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:00.191351 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c12339f-7b1c-4085-83bb-a362a34f659e None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:00.192695 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 122/242] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:32:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 08:32:00.198292 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefcc025-d7e8-4f3d-b786-f8d46858e835 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:00.198638 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefcc025-d7e8-4f3d-b786-f8d46858e835 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:00.198980 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefcc025-d7e8-4f3d-b786-f8d46858e835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:00.210633 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732986b5-b95b-4e65-874a-448e774f4522 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:00.218010 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-aefcc025-d7e8-4f3d-b786-f8d46858e835 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:32:00.219193 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-aefcc025-d7e8-4f3d-b786-f8d46858e835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:32:00.221935 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:00.235075 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:00.246401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb056d18-58a3-4173-9550-fab3f4241203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:00.247099 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb056d18-58a3-4173-9550-fab3f4241203 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:00.247309 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb056d18-58a3-4173-9550-fab3f4241203 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:00.247580 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb056d18-58a3-4173-9550-fab3f4241203 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:00.248665 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb056d18-58a3-4173-9550-fab3f4241203 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:00.250160 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb056d18-58a3-4173-9550-fab3f4241203 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:00.251725 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 121/243] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:32:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:00.252967 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb48e59-7c89-4a01-a73b-63b6b02a3fee None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:00.253252 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb48e59-7c89-4a01-a73b-63b6b02a3fee None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:00.253525 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb48e59-7c89-4a01-a73b-63b6b02a3fee None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:00.254856 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 122/244] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:32:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:32:00.257351 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb48e59-7c89-4a01-a73b-63b6b02a3fee None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:00.289932 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 123/245] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:32:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 08:32:00.291821 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c47497-5c63-442a-93a4-a29f315b5278 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:00.292109 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c47497-5c63-442a-93a4-a29f315b5278 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:00.292381 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c47497-5c63-442a-93a4-a29f315b5278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:00.313609 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-01c47497-5c63-442a-93a4-a29f315b5278 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:32:00.315036 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-01c47497-5c63-442a-93a4-a29f315b5278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:32:00.371313 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb48e59-7c89-4a01-a73b-63b6b02a3fee None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:00.381803 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 124/246] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:32:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 08:32:00.414225 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c09cb6b-835b-4951-97eb-193c352dfce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:00.414841 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c09cb6b-835b-4951-97eb-193c352dfce8 None admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:00.415125 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c09cb6b-835b-4951-97eb-193c352dfce8 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:00.415403 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c09cb6b-835b-4951-97eb-193c352dfce8 None admin] PATH_INFO: `/v3/groups` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:00.416572 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c09cb6b-835b-4951-97eb-193c352dfce8 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:00.418041 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c09cb6b-835b-4951-97eb-193c352dfce8 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:00.435261 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 123/247] 10.222.0.30 () {60 vars in 1134 bytes} [Sat Oct 15 08:32:00 2022] POST /identity/v3/groups => generated 226 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 08:32:00.559757 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c47497-5c63-442a-93a4-a29f315b5278 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:00.634967 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:00.659316 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c09cb6b-835b-4951-97eb-193c352dfce8 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:00.667269 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c47497-5c63-442a-93a4-a29f315b5278 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:00.707581 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c435ec-3db4-4cdb-8070-bee4f672a2e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:00.708517 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c435ec-3db4-4cdb-8070-bee4f672a2e4 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:00.709010 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c435ec-3db4-4cdb-8070-bee4f672a2e4 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:00.709287 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c435ec-3db4-4cdb-8070-bee4f672a2e4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:00.725913 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c435ec-3db4-4cdb-8070-bee4f672a2e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:32:00.730918 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-25c435ec-3db4-4cdb-8070-bee4f672a2e4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 15 08:32:00.730918 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:00.730918 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:00.730918 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:00.730918 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:00.730918 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:00.730918 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:00.730918 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:00.730918 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:00.730918 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:00.730918 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:00.730918 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:00.730918 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:32:00.730918 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:32:00.730918 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:32:00.730918 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:32:00.731831 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:32:00.731831 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:32:00.731831 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:32:00.731831 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:32:00.731831 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:00.731831 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:00.731831 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:00.731831 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:00.731831 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:00.731831 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:00.731831 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:00.731831 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:00.731831 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:00.731831 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:00.731831 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:00.731831 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:00.732619 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:00.732619 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:00.732619 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:00.732619 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:00.732619 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:32:00.732619 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:32:00.732619 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:32:00.732619 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:32:00.732619 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:32:00.732619 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:32:00.732619 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 15 08:32:00.732619 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:00.732619 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 125/248] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 08:32:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:00.734868 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:00.739510 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c435ec-3db4-4cdb-8070-bee4f672a2e4 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:00.795444 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c09cb6b-835b-4951-97eb-193c352dfce8 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:00.851272 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53dcaa6-0124-470c-be05-573f79402a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:00.851959 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53dcaa6-0124-470c-be05-573f79402a07 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:00.852221 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53dcaa6-0124-470c-be05-573f79402a07 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:00.852501 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53dcaa6-0124-470c-be05-573f79402a07 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:00.859878 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c435ec-3db4-4cdb-8070-bee4f672a2e4 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:00.867807 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53dcaa6-0124-470c-be05-573f79402a07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:32:00.874752 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-e53dcaa6-0124-470c-be05-573f79402a07 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 15 08:32:00.874752 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:00.874752 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:00.874752 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:00.874752 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:00.874752 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:00.874752 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:00.874752 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:00.874752 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:00.874752 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:00.874752 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:00.874752 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:00.874752 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:32:00.874752 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:32:00.874752 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:32:00.874752 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:32:00.875671 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:32:00.875671 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:32:00.875671 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:32:00.875671 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:32:00.875671 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:00.875671 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:00.875671 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:00.875671 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:00.875671 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:00.875671 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:00.875671 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:00.875671 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:00.875671 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:00.875671 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:00.875671 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:00.875671 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:00.876485 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:00.876485 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:00.876485 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:00.876485 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:00.876485 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:32:00.876485 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:32:00.876485 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:32:00.876485 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:32:00.876485 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:32:00.876485 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:32:00.876485 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 15 08:32:00.876485 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:00.877146 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 124/249] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 08:32:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 08:32:00.887599 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53dcaa6-0124-470c-be05-573f79402a07 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:00.900610 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:00.908821 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85447b17-c5bb-4189-b717-cf6655f05208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:00.909770 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85447b17-c5bb-4189-b717-cf6655f05208 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:00.910410 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85447b17-c5bb-4189-b717-cf6655f05208 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:00.910684 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85447b17-c5bb-4189-b717-cf6655f05208 None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:00.911349 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85447b17-c5bb-4189-b717-cf6655f05208 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:00.911654 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85447b17-c5bb-4189-b717-cf6655f05208 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:00.913621 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85447b17-c5bb-4189-b717-cf6655f05208 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:00.926241 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 126/250] 10.222.0.30 () {56 vars in 1101 bytes} [Sat Oct 15 08:32:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 08:32:00.931726 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85447b17-c5bb-4189-b717-cf6655f05208 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:01.024666 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53dcaa6-0124-470c-be05-573f79402a07 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:01.041345 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:01.063175 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fa07f1-b889-469b-b998-fe699f548b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:01.063792 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa07f1-b889-469b-b998-fe699f548b33 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:01.064074 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa07f1-b889-469b-b998-fe699f548b33 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:01.064350 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa07f1-b889-469b-b998-fe699f548b33 None admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:01.065821 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fa07f1-b889-469b-b998-fe699f548b33 None admin] RBAC: Adding query filter params (name=member) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:01.066135 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fa07f1-b889-469b-b998-fe699f548b33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:01.067595 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fa07f1-b889-469b-b998-fe699f548b33 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:01.078410 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 125/251] 10.222.0.30 () {56 vars in 1101 bytes} [Sat Oct 15 08:32:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:01.079132 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85447b17-c5bb-4189-b717-cf6655f05208 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:01.084144 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fa07f1-b889-469b-b998-fe699f548b33 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:01.120967 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47156a2c-e4a0-4336-ac0b-009c0de4cbd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:01.121617 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47156a2c-e4a0-4336-ac0b-009c0de4cbd7 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:01.121935 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47156a2c-e4a0-4336-ac0b-009c0de4cbd7 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:01.122216 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47156a2c-e4a0-4336-ac0b-009c0de4cbd7 None admin] PATH_INFO: `/v3/users/8afe315855ea4c18ab8570ce4f861acd` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:01.123519 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47156a2c-e4a0-4336-ac0b-009c0de4cbd7 None admin] RBAC: Authorizing `identity:get_user(user_id=8afe315855ea4c18ab8570ce4f861acd)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:01.125829 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47156a2c-e4a0-4336-ac0b-009c0de4cbd7 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:01.127625 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 127/252] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 08:32:00 2022] GET /identity/v3/users/8afe315855ea4c18ab8570ce4f861acd => generated 293 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 08:32:01.133278 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47156a2c-e4a0-4336-ac0b-009c0de4cbd7 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:01.206418 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fa07f1-b889-469b-b998-fe699f548b33 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:01.215105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:01.249957 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f639cd-1c48-4157-84d7-7abcee8d81f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:01.251420 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f639cd-1c48-4157-84d7-7abcee8d81f2 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:01.251901 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f639cd-1c48-4157-84d7-7abcee8d81f2 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:01.252372 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f639cd-1c48-4157-84d7-7abcee8d81f2 None admin] PATH_INFO: `/v3/users/ebfaacb4deeb4a208cfaf1b6f933b2f4` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:01.254304 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f639cd-1c48-4157-84d7-7abcee8d81f2 None admin] RBAC: Authorizing `identity:get_user(user_id=ebfaacb4deeb4a208cfaf1b6f933b2f4)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:01.256572 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f639cd-1c48-4157-84d7-7abcee8d81f2 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:01.260323 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 126/253] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 08:32:01 2022] GET /identity/v3/users/ebfaacb4deeb4a208cfaf1b6f933b2f4 => generated 293 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:01.265618 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f639cd-1c48-4157-84d7-7abcee8d81f2 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:01.269599 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47156a2c-e4a0-4336-ac0b-009c0de4cbd7 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:01.324354 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eaafef-f443-406f-9e3f-6a428604e5c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:01.325013 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eaafef-f443-406f-9e3f-6a428604e5c7 None admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:01.325298 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eaafef-f443-406f-9e3f-6a428604e5c7 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:01.325577 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eaafef-f443-406f-9e3f-6a428604e5c7 None admin] PATH_INFO: `/v3/system/users/8afe315855ea4c18ab8570ce4f861acd/roles/080a66722a3d4936966a3a50bf9bafe1` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:01.327548 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eaafef-f443-406f-9e3f-6a428604e5c7 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=8afe315855ea4c18ab8570ce4f861acd, role_id=080a66722a3d4936966a3a50bf9bafe1)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:01.329293 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eaafef-f443-406f-9e3f-6a428604e5c7 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:01.334054 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:01.339896 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 128/254] 10.222.0.30 () {58 vars in 1334 bytes} [Sat Oct 15 08:32:01 2022] PUT /identity/v3/system/users/8afe315855ea4c18ab8570ce4f861acd/roles/080a66722a3d4936966a3a50bf9bafe1 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:32:01.421451 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f639cd-1c48-4157-84d7-7abcee8d81f2 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:01.465031 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d093f79-5f9f-4c3c-9768-49e18b9221d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:01.465582 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d093f79-5f9f-4c3c-9768-49e18b9221d6 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:01.465994 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d093f79-5f9f-4c3c-9768-49e18b9221d6 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:01.466268 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d093f79-5f9f-4c3c-9768-49e18b9221d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:01.467075 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d093f79-5f9f-4c3c-9768-49e18b9221d6 None admin] RBAC: Adding query filter params (role.id=3248f0268738489b883041851d681bb7, user.id=ebfaacb4deeb4a208cfaf1b6f933b2f4, scope.system=all) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:01.467389 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d093f79-5f9f-4c3c-9768-49e18b9221d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:01.468909 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d093f79-5f9f-4c3c-9768-49e18b9221d6 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:01.483080 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 127/255] 10.222.0.30 () {56 vars in 1308 bytes} [Sat Oct 15 08:32:01 2022] GET /identity/v3/role_assignments?user.id=ebfaacb4deeb4a208cfaf1b6f933b2f4&scope.system=all&role.id=3248f0268738489b883041851d681bb7 => generated 510 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 08:32:02.934963 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def1331b-29f2-495d-adca-d87b615e6144 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:02.935991 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def1331b-29f2-495d-adca-d87b615e6144 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:02.936481 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def1331b-29f2-495d-adca-d87b615e6144 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:02.938419 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 129/256] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:32:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:32:02.943590 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f9663-3491-4d77-9ada-e6efc7db952b None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:02.944401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f9663-3491-4d77-9ada-e6efc7db952b None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:02.944876 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f9663-3491-4d77-9ada-e6efc7db952b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:02.974647 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-ff1f9663-3491-4d77-9ada-e6efc7db952b None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:32:02.976213 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-ff1f9663-3491-4d77-9ada-e6efc7db952b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:32:03.046368 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 128/257] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:32:02 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 15 08:32:03.245644 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf4bd39-4da7-40e3-8840-9e59a1a0713d None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:03.246331 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf4bd39-4da7-40e3-8840-9e59a1a0713d None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:03.246816 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf4bd39-4da7-40e3-8840-9e59a1a0713d None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:03.248387 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 130/258] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:32:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:32:03.253811 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba14108-117c-42fe-9979-65dfbb7e9b01 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:03.254340 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba14108-117c-42fe-9979-65dfbb7e9b01 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:03.254813 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba14108-117c-42fe-9979-65dfbb7e9b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:03.280905 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-3ba14108-117c-42fe-9979-65dfbb7e9b01 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:32:03.282567 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-3ba14108-117c-42fe-9979-65dfbb7e9b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:32:03.297445 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 129/259] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:32:03 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 15 08:32:03.489713 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf4bd39-4da7-40e3-8840-9e59a1a0713d None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:03.507953 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf4bd39-4da7-40e3-8840-9e59a1a0713d None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:03.513246 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6508fce4-b67f-437d-abe6-b1bcf1192a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:03.514168 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6508fce4-b67f-437d-abe6-b1bcf1192a2c admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:03.514653 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6508fce4-b67f-437d-abe6-b1bcf1192a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:03.515157 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6508fce4-b67f-437d-abe6-b1bcf1192a2c admin admin] PATH_INFO: `/v3/roles/member` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:03.532750 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6508fce4-b67f-437d-abe6-b1bcf1192a2c admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:32:03.537914 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-6508fce4-b67f-437d-abe6-b1bcf1192a2c admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 15 08:32:03.537914 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:03.537914 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:03.537914 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:03.537914 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:03.537914 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:03.537914 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:03.537914 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:03.537914 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:03.537914 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:03.537914 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:03.537914 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:03.537914 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:32:03.537914 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:32:03.537914 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:32:03.537914 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:32:03.538784 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:32:03.538784 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:32:03.538784 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:32:03.538784 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:32:03.538784 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:03.538784 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:03.538784 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:03.538784 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:03.538784 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:03.538784 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:03.538784 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:03.538784 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:03.538784 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:03.538784 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:03.538784 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:03.538784 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:03.539583 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:03.539583 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:03.539583 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:03.539583 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:03.539583 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:32:03.539583 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:32:03.539583 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:32:03.539583 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:32:03.539583 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:32:03.539583 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:32:03.539583 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 15 08:32:03.539583 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:03.542644 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 131/260] 10.222.0.30 () {56 vars in 1120 bytes} [Sat Oct 15 08:32:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 53 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:03.548369 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba14108-117c-42fe-9979-65dfbb7e9b01 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:03.558698 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba14108-117c-42fe-9979-65dfbb7e9b01 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:03.563510 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760d8231-47e3-410f-890e-7b6b1cc71867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:03.564343 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d8231-47e3-410f-890e-7b6b1cc71867 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:03.564824 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d8231-47e3-410f-890e-7b6b1cc71867 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:03.565293 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d8231-47e3-410f-890e-7b6b1cc71867 admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:03.566216 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760d8231-47e3-410f-890e-7b6b1cc71867 admin admin] RBAC: Adding query filter params (name=member) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:03.566737 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760d8231-47e3-410f-890e-7b6b1cc71867 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:03.568726 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: WARNING py.warnings [None req-760d8231-47e3-410f-890e-7b6b1cc71867 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 08:32:03.568726 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: warnings.warn(msg) Oct 15 08:32:03.568726 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: Oct 15 08:32:03.570164 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760d8231-47e3-410f-890e-7b6b1cc71867 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:03.585658 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 130/261] 10.222.0.30 () {56 vars in 1122 bytes} [Sat Oct 15 08:32:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:03.591453 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6508fce4-b67f-437d-abe6-b1bcf1192a2c admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:03.607455 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6508fce4-b67f-437d-abe6-b1bcf1192a2c admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:03.612227 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b5f634-eaf4-4b25-be6e-486bf4727d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:03.613103 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b5f634-eaf4-4b25-be6e-486bf4727d86 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:03.613602 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b5f634-eaf4-4b25-be6e-486bf4727d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:03.614142 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b5f634-eaf4-4b25-be6e-486bf4727d86 admin admin] PATH_INFO: `/v3/groups/bc0af4e23b84458dbca74018fb17d6d2` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:03.616733 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760d8231-47e3-410f-890e-7b6b1cc71867 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:03.625316 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b5f634-eaf4-4b25-be6e-486bf4727d86 admin admin] RBAC: Authorizing `identity:get_group(group_id=bc0af4e23b84458dbca74018fb17d6d2)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:03.625316 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: WARNING py.warnings [None req-45b5f634-eaf4-4b25-be6e-486bf4727d86 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 08:32:03.625316 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: warnings.warn(msg) Oct 15 08:32:03.625316 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: Oct 15 08:32:03.626514 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b5f634-eaf4-4b25-be6e-486bf4727d86 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:03.630394 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 132/262] 10.222.0.30 () {56 vars in 1201 bytes} [Sat Oct 15 08:32:03 2022] GET /identity/v3/groups/bc0af4e23b84458dbca74018fb17d6d2 => generated 226 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:03.632664 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760d8231-47e3-410f-890e-7b6b1cc71867 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:03.635432 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b5f634-eaf4-4b25-be6e-486bf4727d86 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:03.643424 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bfdab0-f6c9-4e59-9b9a-6b03113a2e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:03.644344 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bfdab0-f6c9-4e59-9b9a-6b03113a2e95 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:03.644826 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bfdab0-f6c9-4e59-9b9a-6b03113a2e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:03.645296 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bfdab0-f6c9-4e59-9b9a-6b03113a2e95 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:03.645920 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b5f634-eaf4-4b25-be6e-486bf4727d86 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:03.652910 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d20e11d-1b3b-47b1-8af8-e8daa97df4c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:03.653548 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d20e11d-1b3b-47b1-8af8-e8daa97df4c6 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:03.653853 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d20e11d-1b3b-47b1-8af8-e8daa97df4c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:03.654135 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d20e11d-1b3b-47b1-8af8-e8daa97df4c6 admin admin] PATH_INFO: `/v3/projects/5667e30274a94890a1493a76daa4eced` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:03.655254 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d20e11d-1b3b-47b1-8af8-e8daa97df4c6 admin admin] RBAC: Authorizing `identity:get_project(project_id=5667e30274a94890a1493a76daa4eced)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:03.657316 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d20e11d-1b3b-47b1-8af8-e8daa97df4c6 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:03.658961 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 133/263] 10.222.0.30 () {56 vars in 1207 bytes} [Sat Oct 15 08:32:03 2022] GET /identity/v3/projects/5667e30274a94890a1493a76daa4eced => generated 298 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:03.667057 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d20e11d-1b3b-47b1-8af8-e8daa97df4c6 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:03.678619 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bfdab0-f6c9-4e59-9b9a-6b03113a2e95 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:32:03.684575 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-26bfdab0-f6c9-4e59-9b9a-6b03113a2e95 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 08:32:03.684575 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:03.684575 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:03.684575 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:03.684575 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:03.684575 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:03.684575 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:03.684575 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:03.684575 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:03.684575 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:03.684575 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:03.684575 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:03.684575 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:32:03.684575 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:32:03.684575 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:32:03.684575 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:32:03.685450 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:32:03.685450 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:32:03.685450 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:32:03.685450 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:32:03.685450 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:03.685450 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:03.685450 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:03.685450 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:03.685450 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:03.685450 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:03.685450 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:03.685450 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:03.685450 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:03.685450 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:03.685450 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:03.685450 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:03.686269 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:03.686269 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:03.686269 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:03.686269 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:03.686269 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:32:03.686269 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:32:03.686269 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:32:03.686269 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:32:03.686269 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:32:03.686269 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:32:03.686269 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 08:32:03.686269 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:03.688063 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d20e11d-1b3b-47b1-8af8-e8daa97df4c6 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:03.688759 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 131/264] 10.222.0.30 () {56 vars in 1135 bytes} [Sat Oct 15 08:32:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 72 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:03.694663 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bfdab0-f6c9-4e59-9b9a-6b03113a2e95 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:03.697974 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef75cec3-a5ec-41a6-8afb-0e9fb8b9a994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:03.698646 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75cec3-a5ec-41a6-8afb-0e9fb8b9a994 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:03.698928 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75cec3-a5ec-41a6-8afb-0e9fb8b9a994 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:03.699204 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75cec3-a5ec-41a6-8afb-0e9fb8b9a994 admin admin] PATH_INFO: `/v3/projects/5667e30274a94890a1493a76daa4eced/groups/bc0af4e23b84458dbca74018fb17d6d2/roles/3248f0268738489b883041851d681bb7` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:03.701094 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef75cec3-a5ec-41a6-8afb-0e9fb8b9a994 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5667e30274a94890a1493a76daa4eced, group_id=bc0af4e23b84458dbca74018fb17d6d2, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:03.705722 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: WARNING py.warnings [None req-ef75cec3-a5ec-41a6-8afb-0e9fb8b9a994 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 08:32:03.705722 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: warnings.warn(msg) Oct 15 08:32:03.705722 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: Oct 15 08:32:03.705722 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef75cec3-a5ec-41a6-8afb-0e9fb8b9a994 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:03.711605 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bfdab0-f6c9-4e59-9b9a-6b03113a2e95 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:03.723704 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c1781f-b53e-4e8b-a9ce-d17fc3cea3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:03.724499 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c1781f-b53e-4e8b-a9ce-d17fc3cea3aa admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:03.724916 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c1781f-b53e-4e8b-a9ce-d17fc3cea3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:03.725435 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c1781f-b53e-4e8b-a9ce-d17fc3cea3aa admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:03.726266 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c1781f-b53e-4e8b-a9ce-d17fc3cea3aa admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:03.726628 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c1781f-b53e-4e8b-a9ce-d17fc3cea3aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:03.730908 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c1781f-b53e-4e8b-a9ce-d17fc3cea3aa admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:03.747293 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 132/265] 10.222.0.30 () {56 vars in 1132 bytes} [Sat Oct 15 08:32:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:03.749848 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c1781f-b53e-4e8b-a9ce-d17fc3cea3aa admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:03.751660 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 134/266] 10.222.0.30 () {58 vars in 1463 bytes} [Sat Oct 15 08:32:03 2022] PUT /identity/v3/projects/5667e30274a94890a1493a76daa4eced/groups/bc0af4e23b84458dbca74018fb17d6d2/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:32:03.761379 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c1781f-b53e-4e8b-a9ce-d17fc3cea3aa admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:03.780888 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:03.854601 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc55dd92-8b2e-4b29-bbff-1c07c8850203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:03.855260 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc55dd92-8b2e-4b29-bbff-1c07c8850203 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:03.855593 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc55dd92-8b2e-4b29-bbff-1c07c8850203 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:03.855873 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc55dd92-8b2e-4b29-bbff-1c07c8850203 admin admin] PATH_INFO: `/v3/groups/bc0af4e23b84458dbca74018fb17d6d2` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:03.858719 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc55dd92-8b2e-4b29-bbff-1c07c8850203 admin admin] RBAC: Authorizing `identity:get_group(group_id=bc0af4e23b84458dbca74018fb17d6d2)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:03.859774 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: WARNING py.warnings [None req-dc55dd92-8b2e-4b29-bbff-1c07c8850203 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 08:32:03.859774 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: warnings.warn(msg) Oct 15 08:32:03.859774 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: Oct 15 08:32:03.861011 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc55dd92-8b2e-4b29-bbff-1c07c8850203 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:03.863115 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 133/267] 10.222.0.30 () {56 vars in 1201 bytes} [Sat Oct 15 08:32:03 2022] GET /identity/v3/groups/bc0af4e23b84458dbca74018fb17d6d2 => generated 226 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 08:32:03.870339 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef75cec3-a5ec-41a6-8afb-0e9fb8b9a994 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:03.891180 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef75cec3-a5ec-41a6-8afb-0e9fb8b9a994 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:03.902265 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e2085f-ca75-48e3-a8ff-173f8bbf04a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:03.902927 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e2085f-ca75-48e3-a8ff-173f8bbf04a4 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:03.903209 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e2085f-ca75-48e3-a8ff-173f8bbf04a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:03.903483 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e2085f-ca75-48e3-a8ff-173f8bbf04a4 admin admin] PATH_INFO: `/v3/projects/5667e30274a94890a1493a76daa4eced` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:03.904682 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e2085f-ca75-48e3-a8ff-173f8bbf04a4 admin admin] RBAC: Authorizing `identity:get_project(project_id=5667e30274a94890a1493a76daa4eced)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:03.906900 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e2085f-ca75-48e3-a8ff-173f8bbf04a4 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:03.908857 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 135/268] 10.222.0.30 () {56 vars in 1207 bytes} [Sat Oct 15 08:32:03 2022] GET /identity/v3/projects/5667e30274a94890a1493a76daa4eced => generated 298 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:03.914927 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc55dd92-8b2e-4b29-bbff-1c07c8850203 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:03.925936 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc55dd92-8b2e-4b29-bbff-1c07c8850203 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:03.935086 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700480fb-af95-4326-8665-09723bb34bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:03.935820 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700480fb-af95-4326-8665-09723bb34bf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:03.936105 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700480fb-af95-4326-8665-09723bb34bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:03.936433 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700480fb-af95-4326-8665-09723bb34bf6 admin admin] PATH_INFO: `/v3/projects/5667e30274a94890a1493a76daa4eced/groups/bc0af4e23b84458dbca74018fb17d6d2/roles/d236b691c1434103b73cc4fd0b39fec1` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:03.939403 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700480fb-af95-4326-8665-09723bb34bf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5667e30274a94890a1493a76daa4eced, group_id=bc0af4e23b84458dbca74018fb17d6d2, role_id=d236b691c1434103b73cc4fd0b39fec1)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:03.940512 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: WARNING py.warnings [None req-700480fb-af95-4326-8665-09723bb34bf6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 08:32:03.940512 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: warnings.warn(msg) Oct 15 08:32:03.940512 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: Oct 15 08:32:03.942656 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700480fb-af95-4326-8665-09723bb34bf6 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:03.967501 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 134/269] 10.222.0.30 () {58 vars in 1463 bytes} [Sat Oct 15 08:32:03 2022] PUT /identity/v3/projects/5667e30274a94890a1493a76daa4eced/groups/bc0af4e23b84458dbca74018fb17d6d2/roles/d236b691c1434103b73cc4fd0b39fec1 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:32:04.004243 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816d48e6-ed97-46b4-a434-e20816ba440d None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:04.004892 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816d48e6-ed97-46b4-a434-e20816ba440d None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:04.004892 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816d48e6-ed97-46b4-a434-e20816ba440d None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:04.006225 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 136/270] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:32:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:32:04.023729 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e01a2-b15c-4473-bbad-fbe8ba342c72 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:04.024019 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e01a2-b15c-4473-bbad-fbe8ba342c72 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:04.024205 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e01a2-b15c-4473-bbad-fbe8ba342c72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:04.052706 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-f87e01a2-b15c-4473-bbad-fbe8ba342c72 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:32:04.054055 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-f87e01a2-b15c-4473-bbad-fbe8ba342c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:32:04.130610 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 135/271] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:32:04 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 15 08:32:04.560491 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816d48e6-ed97-46b4-a434-e20816ba440d None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:04.573852 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816d48e6-ed97-46b4-a434-e20816ba440d None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:04.578608 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664686c6-004c-4aad-84d4-88083d423711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:04.579245 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664686c6-004c-4aad-84d4-88083d423711 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:04.579525 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664686c6-004c-4aad-84d4-88083d423711 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:04.579798 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664686c6-004c-4aad-84d4-88083d423711 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:04.593732 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664686c6-004c-4aad-84d4-88083d423711 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:32:04.598602 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-664686c6-004c-4aad-84d4-88083d423711 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 15 08:32:04.598602 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:04.598602 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:04.598602 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:04.598602 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:04.598602 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:04.598602 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:04.598602 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:04.598602 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:04.598602 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:04.598602 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:04.598602 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:04.598602 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:32:04.598602 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:32:04.598602 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:32:04.598602 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:32:04.599500 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:32:04.599500 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:32:04.599500 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:32:04.599500 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:32:04.599500 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:04.599500 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:04.599500 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:04.599500 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:04.599500 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:04.599500 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:04.599500 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:04.599500 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:04.599500 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:04.599500 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:04.599500 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:04.599500 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:04.600295 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:04.600295 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:04.600295 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:04.600295 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:04.600295 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:32:04.600295 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:32:04.600295 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:32:04.600295 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:32:04.600295 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:32:04.600295 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:32:04.600295 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 15 08:32:04.600295 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:04.600295 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 137/272] 10.222.0.30 () {56 vars in 1120 bytes} [Sat Oct 15 08:32:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 40 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:04.607672 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87e01a2-b15c-4473-bbad-fbe8ba342c72 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:04.624143 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87e01a2-b15c-4473-bbad-fbe8ba342c72 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:04.628974 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d65f642-4090-4104-ac03-424007653c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:04.629616 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d65f642-4090-4104-ac03-424007653c15 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:04.629934 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d65f642-4090-4104-ac03-424007653c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:04.630213 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d65f642-4090-4104-ac03-424007653c15 admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:04.630917 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d65f642-4090-4104-ac03-424007653c15 admin admin] RBAC: Adding query filter params (name=member) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:04.631230 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d65f642-4090-4104-ac03-424007653c15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:04.632928 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d65f642-4090-4104-ac03-424007653c15 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:04.647688 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 136/273] 10.222.0.30 () {56 vars in 1122 bytes} [Sat Oct 15 08:32:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:04.653663 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664686c6-004c-4aad-84d4-88083d423711 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:04.667411 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664686c6-004c-4aad-84d4-88083d423711 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:04.677539 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55bc5e6-de8b-4896-afab-7f3248925e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:04.678220 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55bc5e6-de8b-4896-afab-7f3248925e3e admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:04.678500 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55bc5e6-de8b-4896-afab-7f3248925e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:04.678779 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55bc5e6-de8b-4896-afab-7f3248925e3e admin admin] PATH_INFO: `/v3/groups/bc0af4e23b84458dbca74018fb17d6d2` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:04.679905 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55bc5e6-de8b-4896-afab-7f3248925e3e admin admin] RBAC: Authorizing `identity:get_group(group_id=bc0af4e23b84458dbca74018fb17d6d2)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:04.682484 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55bc5e6-de8b-4896-afab-7f3248925e3e admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:04.684231 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 138/274] 10.222.0.30 () {56 vars in 1201 bytes} [Sat Oct 15 08:32:04 2022] GET /identity/v3/groups/bc0af4e23b84458dbca74018fb17d6d2 => generated 226 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:04.690232 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d65f642-4090-4104-ac03-424007653c15 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:04.700702 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d65f642-4090-4104-ac03-424007653c15 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:04.711216 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e35b42f-4955-4f29-8675-eb127f84d53e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:04.711913 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e35b42f-4955-4f29-8675-eb127f84d53e admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:04.712159 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e35b42f-4955-4f29-8675-eb127f84d53e admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:04.712979 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e35b42f-4955-4f29-8675-eb127f84d53e admin admin] PATH_INFO: `/v3/projects/7537ad5b7fef46308a69ce89f36209c2` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:04.714941 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e35b42f-4955-4f29-8675-eb127f84d53e admin admin] RBAC: Authorizing `identity:get_project(project_id=7537ad5b7fef46308a69ce89f36209c2)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:04.723035 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e35b42f-4955-4f29-8675-eb127f84d53e admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:04.725216 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 137/275] 10.222.0.30 () {56 vars in 1207 bytes} [Sat Oct 15 08:32:04 2022] GET /identity/v3/projects/7537ad5b7fef46308a69ce89f36209c2 => generated 302 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:04.730832 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55bc5e6-de8b-4896-afab-7f3248925e3e admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:04.747447 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55bc5e6-de8b-4896-afab-7f3248925e3e admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:04.757573 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762e1a8-a986-45e5-a972-491b0c19aad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:04.758296 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762e1a8-a986-45e5-a972-491b0c19aad9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:04.758579 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762e1a8-a986-45e5-a972-491b0c19aad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:04.758853 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762e1a8-a986-45e5-a972-491b0c19aad9 admin admin] PATH_INFO: `/v3/projects/7537ad5b7fef46308a69ce89f36209c2/groups/bc0af4e23b84458dbca74018fb17d6d2/roles/3248f0268738489b883041851d681bb7` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:04.760982 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f762e1a8-a986-45e5-a972-491b0c19aad9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7537ad5b7fef46308a69ce89f36209c2, group_id=bc0af4e23b84458dbca74018fb17d6d2, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:04.763816 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f762e1a8-a986-45e5-a972-491b0c19aad9 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:04.790537 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 139/276] 10.222.0.30 () {58 vars in 1463 bytes} [Sat Oct 15 08:32:04 2022] PUT /identity/v3/projects/7537ad5b7fef46308a69ce89f36209c2/groups/bc0af4e23b84458dbca74018fb17d6d2/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:32:04.897793 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3360e7fa-25f0-4976-b6a9-e377684b7ed0 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:04.898143 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3360e7fa-25f0-4976-b6a9-e377684b7ed0 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:04.898339 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3360e7fa-25f0-4976-b6a9-e377684b7ed0 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:04.900167 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 138/277] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:32:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:32:04.905532 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ac7d2-5437-4165-89ad-036c5570cf81 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:04.905791 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ac7d2-5437-4165-89ad-036c5570cf81 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:04.906079 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ac7d2-5437-4165-89ad-036c5570cf81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:04.913149 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2181880-394c-4205-a710-00bb346e439b None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:04.913704 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2181880-394c-4205-a710-00bb346e439b None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:04.913998 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2181880-394c-4205-a710-00bb346e439b None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:04.916074 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 139/278] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:32:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 08:32:04.921697 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b34680-d8c9-4b39-8209-8dd7a754c9b4 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:04.921969 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b34680-d8c9-4b39-8209-8dd7a754c9b4 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:04.922251 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b34680-d8c9-4b39-8209-8dd7a754c9b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:04.934578 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-b96ac7d2-5437-4165-89ad-036c5570cf81 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:32:04.935812 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-b96ac7d2-5437-4165-89ad-036c5570cf81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:32:04.952759 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-93b34680-d8c9-4b39-8209-8dd7a754c9b4 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:32:04.954038 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-93b34680-d8c9-4b39-8209-8dd7a754c9b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:32:05.061394 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 140/279] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:32:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 08:32:05.111147 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 140/280] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:32:04 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 15 08:32:05.390606 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b34680-d8c9-4b39-8209-8dd7a754c9b4 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:05.399423 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:05.528507 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b34680-d8c9-4b39-8209-8dd7a754c9b4 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:05.569364 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cd1ac9-ee9c-4bf6-a68c-a7559dc5db66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:05.570041 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cd1ac9-ee9c-4bf6-a68c-a7559dc5db66 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:05.570326 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cd1ac9-ee9c-4bf6-a68c-a7559dc5db66 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:05.570600 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cd1ac9-ee9c-4bf6-a68c-a7559dc5db66 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:05.581672 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cd1ac9-ee9c-4bf6-a68c-a7559dc5db66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:32:05.586492 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-22cd1ac9-ee9c-4bf6-a68c-a7559dc5db66 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 15 08:32:05.586492 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:05.586492 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:05.586492 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:05.586492 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:05.586492 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:05.586492 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:05.586492 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:05.586492 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:05.586492 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:05.586492 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:05.586492 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:05.586492 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:32:05.586492 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:32:05.586492 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:32:05.586492 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:32:05.588545 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:32:05.588545 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:32:05.588545 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:32:05.588545 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:32:05.588545 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:05.588545 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:05.588545 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:05.588545 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:05.588545 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:05.588545 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:05.588545 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:05.588545 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:05.588545 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:05.588545 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:05.588545 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:05.588545 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:05.590484 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:05.590484 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:05.590484 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:05.590484 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:05.590484 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:32:05.590484 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:32:05.590484 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:32:05.590484 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:32:05.590484 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:32:05.590484 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:32:05.590484 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 15 08:32:05.590484 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:05.590484 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 141/281] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 08:32:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:05.594783 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96ac7d2-5437-4165-89ad-036c5570cf81 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:05.597322 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:05.752437 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96ac7d2-5437-4165-89ad-036c5570cf81 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:05.756854 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cd1ac9-ee9c-4bf6-a68c-a7559dc5db66 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:05.777490 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cd1ac9-ee9c-4bf6-a68c-a7559dc5db66 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:05.786427 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa9e623-a07e-4b70-b8e6-82c7fffc4534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:05.787073 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa9e623-a07e-4b70-b8e6-82c7fffc4534 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:05.787354 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa9e623-a07e-4b70-b8e6-82c7fffc4534 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:05.787625 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa9e623-a07e-4b70-b8e6-82c7fffc4534 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:05.799455 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cdfac4-cb97-4b29-a2c6-8a9555fea3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:05.800197 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cdfac4-cb97-4b29-a2c6-8a9555fea3cb None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:05.800491 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cdfac4-cb97-4b29-a2c6-8a9555fea3cb None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:05.800776 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cdfac4-cb97-4b29-a2c6-8a9555fea3cb None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:05.801457 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cdfac4-cb97-4b29-a2c6-8a9555fea3cb None admin] RBAC: Adding query filter params (name=reader) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:05.801801 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cdfac4-cb97-4b29-a2c6-8a9555fea3cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:05.803200 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cdfac4-cb97-4b29-a2c6-8a9555fea3cb None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:05.803375 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa9e623-a07e-4b70-b8e6-82c7fffc4534 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:32:05.812307 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-caa9e623-a07e-4b70-b8e6-82c7fffc4534 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 08:32:05.812307 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:05.812307 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:05.812307 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:05.812307 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:05.812307 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:05.812307 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:05.812307 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:05.812307 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:05.812307 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:05.812307 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:05.812307 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:05.812307 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:32:05.812307 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:32:05.812307 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:32:05.812307 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:32:05.814523 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:32:05.814523 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:32:05.814523 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:32:05.814523 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:32:05.814523 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:05.814523 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:05.814523 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:05.814523 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:05.814523 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:05.814523 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:05.814523 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:05.814523 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:05.814523 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:05.814523 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:05.814523 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:05.814523 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:05.816458 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:05.816458 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:05.816458 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:05.816458 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:05.816458 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:32:05.816458 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:32:05.816458 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:32:05.816458 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:32:05.816458 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:32:05.816458 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:32:05.816458 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 08:32:05.816458 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:05.816458 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 142/282] 10.222.0.30 () {56 vars in 1135 bytes} [Sat Oct 15 08:32:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 57 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:05.820457 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 141/283] 10.222.0.30 () {56 vars in 1101 bytes} [Sat Oct 15 08:32:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:05.822406 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa9e623-a07e-4b70-b8e6-82c7fffc4534 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:05.837883 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa9e623-a07e-4b70-b8e6-82c7fffc4534 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:05.839862 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cdfac4-cb97-4b29-a2c6-8a9555fea3cb None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:05.862406 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d695d16f-233e-45b7-a1c8-f63e6db8cb76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:05.863041 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d695d16f-233e-45b7-a1c8-f63e6db8cb76 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:05.863327 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d695d16f-233e-45b7-a1c8-f63e6db8cb76 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:05.863607 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d695d16f-233e-45b7-a1c8-f63e6db8cb76 admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:05.864285 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d695d16f-233e-45b7-a1c8-f63e6db8cb76 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:05.864592 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d695d16f-233e-45b7-a1c8-f63e6db8cb76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:05.866256 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d695d16f-233e-45b7-a1c8-f63e6db8cb76 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:05.881804 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 143/284] 10.222.0.30 () {56 vars in 1132 bytes} [Sat Oct 15 08:32:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:05.900111 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d695d16f-233e-45b7-a1c8-f63e6db8cb76 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:05.915376 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d695d16f-233e-45b7-a1c8-f63e6db8cb76 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:05.926313 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc293780-18e0-4ed1-b7ab-cf353f2623aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:05.926972 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc293780-18e0-4ed1-b7ab-cf353f2623aa admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:05.927259 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc293780-18e0-4ed1-b7ab-cf353f2623aa admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:05.927539 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc293780-18e0-4ed1-b7ab-cf353f2623aa admin admin] PATH_INFO: `/v3/groups/bc0af4e23b84458dbca74018fb17d6d2` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:05.928939 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc293780-18e0-4ed1-b7ab-cf353f2623aa admin admin] RBAC: Authorizing `identity:get_group(group_id=bc0af4e23b84458dbca74018fb17d6d2)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:05.929253 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:05.930891 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc293780-18e0-4ed1-b7ab-cf353f2623aa admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:05.934420 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 144/285] 10.222.0.30 () {56 vars in 1201 bytes} [Sat Oct 15 08:32:05 2022] GET /identity/v3/groups/bc0af4e23b84458dbca74018fb17d6d2 => generated 226 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:05.941427 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc293780-18e0-4ed1-b7ab-cf353f2623aa admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:05.955668 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc293780-18e0-4ed1-b7ab-cf353f2623aa admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:05.970857 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aa2d2f-3199-4c62-a9e1-34a7a8146ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:05.971500 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa2d2f-3199-4c62-a9e1-34a7a8146ede admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:05.971782 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa2d2f-3199-4c62-a9e1-34a7a8146ede admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:05.972058 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa2d2f-3199-4c62-a9e1-34a7a8146ede admin admin] PATH_INFO: `/v3/projects/7537ad5b7fef46308a69ce89f36209c2` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:05.974315 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cdfac4-cb97-4b29-a2c6-8a9555fea3cb None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:05.978419 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aa2d2f-3199-4c62-a9e1-34a7a8146ede admin admin] RBAC: Authorizing `identity:get_project(project_id=7537ad5b7fef46308a69ce89f36209c2)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:05.980508 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aa2d2f-3199-4c62-a9e1-34a7a8146ede admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:05.982672 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 145/286] 10.222.0.30 () {56 vars in 1207 bytes} [Sat Oct 15 08:32:05 2022] GET /identity/v3/projects/7537ad5b7fef46308a69ce89f36209c2 => generated 302 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:05.988030 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aa2d2f-3199-4c62-a9e1-34a7a8146ede admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:06.003891 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aa2d2f-3199-4c62-a9e1-34a7a8146ede admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:06.015296 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb11dcc-745b-470f-acde-c74dff1688ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:06.015943 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb11dcc-745b-470f-acde-c74dff1688ff None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:06.016232 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb11dcc-745b-470f-acde-c74dff1688ff None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:06.016515 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb11dcc-745b-470f-acde-c74dff1688ff None admin] PATH_INFO: `/v3/users/8afe315855ea4c18ab8570ce4f861acd` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:06.017930 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2334a536-cd39-48bb-ad28-9796f58b91d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:06.018594 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2334a536-cd39-48bb-ad28-9796f58b91d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:06.018872 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2334a536-cd39-48bb-ad28-9796f58b91d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:06.019148 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2334a536-cd39-48bb-ad28-9796f58b91d9 admin admin] PATH_INFO: `/v3/projects/7537ad5b7fef46308a69ce89f36209c2/groups/bc0af4e23b84458dbca74018fb17d6d2/roles/d236b691c1434103b73cc4fd0b39fec1` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:06.020270 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb11dcc-745b-470f-acde-c74dff1688ff None admin] RBAC: Authorizing `identity:get_user(user_id=8afe315855ea4c18ab8570ce4f861acd)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:06.021732 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb11dcc-745b-470f-acde-c74dff1688ff None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:06.023498 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 142/287] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 08:32:05 2022] GET /identity/v3/users/8afe315855ea4c18ab8570ce4f861acd => generated 293 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 08:32:06.026513 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2334a536-cd39-48bb-ad28-9796f58b91d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7537ad5b7fef46308a69ce89f36209c2, group_id=bc0af4e23b84458dbca74018fb17d6d2, role_id=d236b691c1434103b73cc4fd0b39fec1)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:06.029189 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2334a536-cd39-48bb-ad28-9796f58b91d9 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:06.032449 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb11dcc-745b-470f-acde-c74dff1688ff None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:06.053895 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 146/288] 10.222.0.30 () {58 vars in 1463 bytes} [Sat Oct 15 08:32:05 2022] PUT /identity/v3/projects/7537ad5b7fef46308a69ce89f36209c2/groups/bc0af4e23b84458dbca74018fb17d6d2/roles/d236b691c1434103b73cc4fd0b39fec1 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 08:32:06.161951 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb11dcc-745b-470f-acde-c74dff1688ff None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:06.198094 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:06.204152 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0652d84-1025-45f6-8e8d-70cf32aee9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:06.204986 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0652d84-1025-45f6-8e8d-70cf32aee9f2 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:06.205522 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0652d84-1025-45f6-8e8d-70cf32aee9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:06.205919 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0652d84-1025-45f6-8e8d-70cf32aee9f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:06.206629 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0652d84-1025-45f6-8e8d-70cf32aee9f2 None admin] RBAC: Adding query filter params (role.id=080a66722a3d4936966a3a50bf9bafe1, user.id=8afe315855ea4c18ab8570ce4f861acd, scope.system=all) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:06.206949 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0652d84-1025-45f6-8e8d-70cf32aee9f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:06.208489 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0652d84-1025-45f6-8e8d-70cf32aee9f2 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:06.225798 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 143/289] 10.222.0.30 () {56 vars in 1308 bytes} [Sat Oct 15 08:32:06 2022] GET /identity/v3/role_assignments?user.id=8afe315855ea4c18ab8570ce4f861acd&scope.system=all&role.id=080a66722a3d4936966a3a50bf9bafe1 => generated 510 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:06.712544 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf547a28-7dd3-4e9a-ab1e-fb5046804a88 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:06.713211 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf547a28-7dd3-4e9a-ab1e-fb5046804a88 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:06.713211 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf547a28-7dd3-4e9a-ab1e-fb5046804a88 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:06.714413 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 147/290] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:32:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:32:06.731213 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a0628-ab24-4f68-94e2-47577b7c04c7 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:06.731511 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a0628-ab24-4f68-94e2-47577b7c04c7 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:06.731767 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a0628-ab24-4f68-94e2-47577b7c04c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:06.761905 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-267a0628-ab24-4f68-94e2-47577b7c04c7 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:32:06.763129 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-267a0628-ab24-4f68-94e2-47577b7c04c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:32:06.830684 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 144/291] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:32:06 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 15 08:32:07.137354 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf547a28-7dd3-4e9a-ab1e-fb5046804a88 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:07.150364 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf547a28-7dd3-4e9a-ab1e-fb5046804a88 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:07.154773 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754af2ce-76aa-40cc-a35c-06df44a73431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:07.155671 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754af2ce-76aa-40cc-a35c-06df44a73431 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:07.156157 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754af2ce-76aa-40cc-a35c-06df44a73431 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:07.156655 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754af2ce-76aa-40cc-a35c-06df44a73431 admin admin] PATH_INFO: `/v3/roles/admin` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:07.168378 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754af2ce-76aa-40cc-a35c-06df44a73431 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:32:07.173973 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-754af2ce-76aa-40cc-a35c-06df44a73431 admin admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 08:32:07.173973 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:07.173973 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:07.173973 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:07.173973 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:07.173973 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:07.173973 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:07.173973 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:07.173973 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:07.173973 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:07.173973 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:07.173973 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:07.173973 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:32:07.173973 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:32:07.173973 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:32:07.173973 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:32:07.174854 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:32:07.174854 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:32:07.174854 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:32:07.174854 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:32:07.174854 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:07.174854 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:07.174854 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:07.174854 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:07.174854 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:07.174854 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:07.174854 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:07.174854 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:07.174854 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:07.174854 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:07.174854 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:07.174854 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:07.175640 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:07.175640 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:07.175640 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:07.175640 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:07.175640 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:32:07.175640 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:32:07.175640 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:32:07.175640 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:32:07.175640 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:32:07.175640 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:32:07.175640 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 08:32:07.175640 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:07.179039 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 148/292] 10.222.0.30 () {56 vars in 1117 bytes} [Sat Oct 15 08:32:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 42 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:07.187443 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267a0628-ab24-4f68-94e2-47577b7c04c7 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:07.198268 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267a0628-ab24-4f68-94e2-47577b7c04c7 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:07.204916 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7161a6cf-c529-4bc2-9b9c-8683ed895017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:07.205857 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161a6cf-c529-4bc2-9b9c-8683ed895017 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:07.206354 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161a6cf-c529-4bc2-9b9c-8683ed895017 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:07.206855 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161a6cf-c529-4bc2-9b9c-8683ed895017 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:07.207725 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7161a6cf-c529-4bc2-9b9c-8683ed895017 admin admin] RBAC: Adding query filter params (name=admin) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:07.208233 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7161a6cf-c529-4bc2-9b9c-8683ed895017 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:07.209441 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: WARNING py.warnings [None req-7161a6cf-c529-4bc2-9b9c-8683ed895017 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 08:32:07.209441 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: warnings.warn(msg) Oct 15 08:32:07.209441 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: Oct 15 08:32:07.210781 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7161a6cf-c529-4bc2-9b9c-8683ed895017 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:07.216706 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52a3833-b515-4771-9d64-90f6150d01e7 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:07.217326 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52a3833-b515-4771-9d64-90f6150d01e7 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:07.217840 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52a3833-b515-4771-9d64-90f6150d01e7 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:07.219440 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 149/293] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:32:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:32:07.224663 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d9489-cd7a-412d-9470-f7238eaf0108 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:07.225180 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d9489-cd7a-412d-9470-f7238eaf0108 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:07.225663 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d9489-cd7a-412d-9470-f7238eaf0108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:07.231788 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 145/294] 10.222.0.30 () {56 vars in 1120 bytes} [Sat Oct 15 08:32:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:07.238681 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7161a6cf-c529-4bc2-9b9c-8683ed895017 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:07.252517 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-2d2d9489-cd7a-412d-9470-f7238eaf0108 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:32:07.254086 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-2d2d9489-cd7a-412d-9470-f7238eaf0108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:32:07.259030 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7161a6cf-c529-4bc2-9b9c-8683ed895017 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:07.263171 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:07.274347 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee389815-b5f9-4222-bd4b-75c42e25f0fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:07.275027 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee389815-b5f9-4222-bd4b-75c42e25f0fa admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:07.275308 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee389815-b5f9-4222-bd4b-75c42e25f0fa admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:07.275585 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee389815-b5f9-4222-bd4b-75c42e25f0fa admin admin] PATH_INFO: `/v3/groups/209444e529574ef1a5d5d1a0075cb242` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:07.282365 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee389815-b5f9-4222-bd4b-75c42e25f0fa admin admin] RBAC: Authorizing `identity:get_group(group_id=209444e529574ef1a5d5d1a0075cb242)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:07.284518 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee389815-b5f9-4222-bd4b-75c42e25f0fa admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:07.286387 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 146/295] 10.222.0.30 () {56 vars in 1201 bytes} [Sat Oct 15 08:32:07 2022] GET /identity/v3/groups/209444e529574ef1a5d5d1a0075cb242 => generated 229 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 08:32:07.296208 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee389815-b5f9-4222-bd4b-75c42e25f0fa admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:07.317274 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee389815-b5f9-4222-bd4b-75c42e25f0fa admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:07.331488 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401b1b9f-4bff-4c28-adac-416b89857142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:07.332149 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b1b9f-4bff-4c28-adac-416b89857142 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:07.332486 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b1b9f-4bff-4c28-adac-416b89857142 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:07.332772 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b1b9f-4bff-4c28-adac-416b89857142 admin admin] PATH_INFO: `/v3/projects/d200ced7ea7746bab37279a9888da771` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:07.334078 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401b1b9f-4bff-4c28-adac-416b89857142 admin admin] RBAC: Authorizing `identity:get_project(project_id=d200ced7ea7746bab37279a9888da771)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:07.338393 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401b1b9f-4bff-4c28-adac-416b89857142 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:07.341371 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 147/296] 10.222.0.30 () {56 vars in 1207 bytes} [Sat Oct 15 08:32:07 2022] GET /identity/v3/projects/d200ced7ea7746bab37279a9888da771 => generated 344 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:07.349849 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401b1b9f-4bff-4c28-adac-416b89857142 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:07.369396 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401b1b9f-4bff-4c28-adac-416b89857142 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:07.371747 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 150/297] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:32:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 08:32:07.377244 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772e2843-3a98-4d5a-83d4-a1b3f6972bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:07.378247 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772e2843-3a98-4d5a-83d4-a1b3f6972bc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:07.378829 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772e2843-3a98-4d5a-83d4-a1b3f6972bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:07.379304 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772e2843-3a98-4d5a-83d4-a1b3f6972bc9 admin admin] PATH_INFO: `/v3/projects/d200ced7ea7746bab37279a9888da771/groups/209444e529574ef1a5d5d1a0075cb242/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:07.381436 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772e2843-3a98-4d5a-83d4-a1b3f6972bc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d200ced7ea7746bab37279a9888da771, group_id=209444e529574ef1a5d5d1a0075cb242, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:07.404439 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772e2843-3a98-4d5a-83d4-a1b3f6972bc9 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:07.423983 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 148/298] 10.222.0.30 () {58 vars in 1463 bytes} [Sat Oct 15 08:32:07 2022] PUT /identity/v3/projects/d200ced7ea7746bab37279a9888da771/groups/209444e529574ef1a5d5d1a0075cb242/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:32:07.478831 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3c1255-879e-462e-bede-f648b12e8342 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:07.479118 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3c1255-879e-462e-bede-f648b12e8342 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:07.479562 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3c1255-879e-462e-bede-f648b12e8342 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:07.481257 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 151/299] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:32:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:32:07.489075 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efaacdd-2a94-4083-999b-136ec3cae84a None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:07.489362 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efaacdd-2a94-4083-999b-136ec3cae84a None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:07.489640 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efaacdd-2a94-4083-999b-136ec3cae84a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:07.512820 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-6efaacdd-2a94-4083-999b-136ec3cae84a None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:32:07.514027 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-6efaacdd-2a94-4083-999b-136ec3cae84a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:32:07.608316 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 149/300] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:32:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 08:32:07.788321 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3c1255-879e-462e-bede-f648b12e8342 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:07.906768 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3c1255-879e-462e-bede-f648b12e8342 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:07.957840 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28d04f5-1b97-42ea-aa1f-9b012800afd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:07.958734 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d04f5-1b97-42ea-aa1f-9b012800afd3 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:07.959189 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d04f5-1b97-42ea-aa1f-9b012800afd3 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:07.959609 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d04f5-1b97-42ea-aa1f-9b012800afd3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:07.977009 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28d04f5-1b97-42ea-aa1f-9b012800afd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:32:07.982735 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-b28d04f5-1b97-42ea-aa1f-9b012800afd3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 08:32:07.982735 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:07.982735 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:07.982735 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:07.982735 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:07.982735 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:07.982735 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:07.982735 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:07.982735 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:07.982735 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:07.982735 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:07.982735 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:07.982735 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:32:07.982735 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:32:07.982735 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:32:07.982735 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:32:07.983838 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:32:07.983838 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:32:07.983838 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:32:07.983838 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:32:07.983838 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:07.983838 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:07.983838 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:07.983838 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:07.983838 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:07.983838 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:07.983838 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:07.983838 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:07.983838 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:07.983838 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:07.983838 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:07.983838 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:07.984801 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:07.984801 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:07.984801 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:07.984801 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:07.984801 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:32:07.984801 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:32:07.984801 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:32:07.984801 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:32:07.984801 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:32:07.984801 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:32:07.984801 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 08:32:07.984801 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:07.987014 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 152/301] 10.222.0.30 () {56 vars in 1114 bytes} [Sat Oct 15 08:32:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:07.992718 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efaacdd-2a94-4083-999b-136ec3cae84a None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:08.055868 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:08.103944 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28d04f5-1b97-42ea-aa1f-9b012800afd3 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:08.108029 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:08.162507 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efaacdd-2a94-4083-999b-136ec3cae84a None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:08.200811 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a4f6a6-dd26-4df8-80cf-2259542ab43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:08.202129 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a4f6a6-dd26-4df8-80cf-2259542ab43b None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:08.202405 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a4f6a6-dd26-4df8-80cf-2259542ab43b None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:08.202828 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a4f6a6-dd26-4df8-80cf-2259542ab43b None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:08.203655 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a4f6a6-dd26-4df8-80cf-2259542ab43b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:08.204123 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a4f6a6-dd26-4df8-80cf-2259542ab43b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:08.205652 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a4f6a6-dd26-4df8-80cf-2259542ab43b None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:08.214012 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 150/302] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 08:32:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:08.216460 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad6301d-d127-4c1c-b63a-5cd99bac6843 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:08.217008 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad6301d-d127-4c1c-b63a-5cd99bac6843 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:08.217515 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad6301d-d127-4c1c-b63a-5cd99bac6843 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:08.218988 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 151/303] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:32:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:32:08.224638 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc65db-17e0-4b8b-a065-01262d7036bf None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:08.225144 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc65db-17e0-4b8b-a065-01262d7036bf None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:08.225605 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc65db-17e0-4b8b-a065-01262d7036bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:08.227529 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28d04f5-1b97-42ea-aa1f-9b012800afd3 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:08.245967 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-03fc65db-17e0-4b8b-a065-01262d7036bf None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:32:08.247330 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-03fc65db-17e0-4b8b-a065-01262d7036bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:32:08.268041 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2765294d-f4f6-4313-88a5-4b4398e0d252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:08.268943 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2765294d-f4f6-4313-88a5-4b4398e0d252 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:08.269393 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2765294d-f4f6-4313-88a5-4b4398e0d252 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:08.269956 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2765294d-f4f6-4313-88a5-4b4398e0d252 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:08.282814 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2765294d-f4f6-4313-88a5-4b4398e0d252 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:32:08.289914 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-2765294d-f4f6-4313-88a5-4b4398e0d252 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 15 08:32:08.289914 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:08.289914 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:08.289914 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:08.289914 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:08.289914 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:08.289914 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:08.289914 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:08.289914 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:08.289914 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:08.289914 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:08.289914 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:08.289914 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:32:08.289914 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:32:08.289914 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:32:08.289914 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:32:08.291011 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:32:08.291011 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:32:08.291011 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:32:08.291011 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:32:08.291011 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:08.291011 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:08.291011 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:08.291011 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:08.291011 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:08.291011 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:08.291011 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:08.291011 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:08.291011 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:08.291011 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:08.291011 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:08.291011 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:08.291952 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:08.291952 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:08.291952 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:08.291952 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:08.291952 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:32:08.291952 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:32:08.291952 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:32:08.291952 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:32:08.291952 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:32:08.291952 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:32:08.291952 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 15 08:32:08.291952 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:08.294299 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 153/304] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 08:32:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 08:32:08.295357 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2765294d-f4f6-4313-88a5-4b4398e0d252 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:08.333476 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 152/305] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:32:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 08:32:08.334466 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fc65db-17e0-4b8b-a065-01262d7036bf None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:08.372087 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:08.375516 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:08.445722 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2765294d-f4f6-4313-88a5-4b4398e0d252 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:08.496373 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fc65db-17e0-4b8b-a065-01262d7036bf None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:08.517956 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ac1277-887a-48e9-afef-a693574a9da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:08.518576 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ac1277-887a-48e9-afef-a693574a9da1 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:08.518858 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ac1277-887a-48e9-afef-a693574a9da1 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:08.519136 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ac1277-887a-48e9-afef-a693574a9da1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:08.563996 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efd4620-8d02-4750-bb4e-0542a88efebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:08.564620 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd4620-8d02-4750-bb4e-0542a88efebf None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:08.564905 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd4620-8d02-4750-bb4e-0542a88efebf None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:08.566170 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd4620-8d02-4750-bb4e-0542a88efebf None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:08.566895 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efd4620-8d02-4750-bb4e-0542a88efebf None admin] RBAC: Adding query filter params (name=member) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:08.567212 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efd4620-8d02-4750-bb4e-0542a88efebf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:08.571140 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efd4620-8d02-4750-bb4e-0542a88efebf None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:08.588522 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 153/306] 10.222.0.30 () {56 vars in 1101 bytes} [Sat Oct 15 08:32:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:08.595475 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efd4620-8d02-4750-bb4e-0542a88efebf None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:08.652054 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ac1277-887a-48e9-afef-a693574a9da1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:08.654672 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ac1277-887a-48e9-afef-a693574a9da1 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:08.744847 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:08.751814 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:08.762235 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efd4620-8d02-4750-bb4e-0542a88efebf None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:08.804300 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d417edb3-846e-47f2-b5a9-3c8a969d52ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:08.805222 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d417edb3-846e-47f2-b5a9-3c8a969d52ae None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:08.805511 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d417edb3-846e-47f2-b5a9-3c8a969d52ae None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:08.806200 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d417edb3-846e-47f2-b5a9-3c8a969d52ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:08.813189 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 154/307] 10.222.0.30 () {58 vars in 1282 bytes} [Sat Oct 15 08:32:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 08:32:08.814082 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ac1277-887a-48e9-afef-a693574a9da1 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:08.908115 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d417edb3-846e-47f2-b5a9-3c8a969d52ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:08.909498 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d417edb3-846e-47f2-b5a9-3c8a969d52ae None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:08.948407 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ac1277-887a-48e9-afef-a693574a9da1 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:08.992846 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0ad68e-e894-4b84-8917-c4a67a31e38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:08.994274 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0ad68e-e894-4b84-8917-c4a67a31e38b None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:08.994562 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0ad68e-e894-4b84-8917-c4a67a31e38b None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:08.994841 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0ad68e-e894-4b84-8917-c4a67a31e38b None admin] PATH_INFO: `/v3/roles/member` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:09.007554 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0ad68e-e894-4b84-8917-c4a67a31e38b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:32:09.010483 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:09.014390 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-2b0ad68e-e894-4b84-8917-c4a67a31e38b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 15 08:32:09.014390 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:09.014390 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:09.014390 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:09.014390 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:09.014390 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:09.014390 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:09.014390 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:09.014390 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:09.014390 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:09.014390 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:09.014390 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:09.014390 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:32:09.014390 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:32:09.014390 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:32:09.014390 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:32:09.016475 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:32:09.016475 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:32:09.016475 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:32:09.016475 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:32:09.016475 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:09.016475 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:09.016475 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:09.016475 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:09.016475 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:09.016475 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:09.016475 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:09.016475 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:09.016475 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:09.016475 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:09.016475 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:09.016475 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:09.018826 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:09.018826 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:09.018826 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:09.018826 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:09.018826 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:32:09.018826 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:32:09.018826 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:32:09.018826 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:32:09.018826 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:32:09.018826 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:32:09.018826 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 15 08:32:09.018826 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:09.018826 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 155/308] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 08:32:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:09.018826 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0ad68e-e894-4b84-8917-c4a67a31e38b None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:09.032985 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 154/309] 10.222.0.30 () {58 vars in 1282 bytes} [Sat Oct 15 08:32:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 15 08:32:09.033946 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d417edb3-846e-47f2-b5a9-3c8a969d52ae None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:09.049058 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:09.151214 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0ad68e-e894-4b84-8917-c4a67a31e38b None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:09.154950 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d417edb3-846e-47f2-b5a9-3c8a969d52ae None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:09.190329 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfd9df6-800c-40fc-817f-21aa459acac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:09.190893 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd9df6-800c-40fc-817f-21aa459acac1 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:09.191177 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd9df6-800c-40fc-817f-21aa459acac1 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:09.191455 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd9df6-800c-40fc-817f-21aa459acac1 None admin] PATH_INFO: `/v3/projects/5667e30274a94890a1493a76daa4eced` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:09.192589 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfd9df6-800c-40fc-817f-21aa459acac1 None admin] RBAC: Authorizing `identity:get_project(project_id=5667e30274a94890a1493a76daa4eced)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:09.194094 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfd9df6-800c-40fc-817f-21aa459acac1 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:09.197823 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 156/310] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 08:32:09 2022] GET /identity/v3/projects/5667e30274a94890a1493a76daa4eced => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:09.197823 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfd9df6-800c-40fc-817f-21aa459acac1 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:09.198641 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e1428c-f587-475f-aaa9-d9c2855db28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:09.199398 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e1428c-f587-475f-aaa9-d9c2855db28c None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:09.199806 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e1428c-f587-475f-aaa9-d9c2855db28c None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:09.200295 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e1428c-f587-475f-aaa9-d9c2855db28c None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:09.201221 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e1428c-f587-475f-aaa9-d9c2855db28c None admin] RBAC: Adding query filter params (name=member) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:09.201789 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e1428c-f587-475f-aaa9-d9c2855db28c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:09.203015 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e1428c-f587-475f-aaa9-d9c2855db28c None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:09.213345 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 155/311] 10.222.0.30 () {56 vars in 1101 bytes} [Sat Oct 15 08:32:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:09.215357 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f195ce0-e599-405d-9e4a-ac8c692abb78 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:09.215652 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f195ce0-e599-405d-9e4a-ac8c692abb78 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:09.215931 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f195ce0-e599-405d-9e4a-ac8c692abb78 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:09.217423 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 156/312] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:32:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:32:09.218191 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f195ce0-e599-405d-9e4a-ac8c692abb78 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:09.283733 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:09.317971 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:09.327127 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfd9df6-800c-40fc-817f-21aa459acac1 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:09.361757 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f195ce0-e599-405d-9e4a-ac8c692abb78 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:09.379244 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db19ff82-4ee2-497e-9ad2-78f03d9824b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:09.379885 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db19ff82-4ee2-497e-9ad2-78f03d9824b4 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:09.380176 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db19ff82-4ee2-497e-9ad2-78f03d9824b4 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:09.380457 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db19ff82-4ee2-497e-9ad2-78f03d9824b4 None admin] PATH_INFO: `/v3/projects/5667e30274a94890a1493a76daa4eced` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:09.381578 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db19ff82-4ee2-497e-9ad2-78f03d9824b4 None admin] RBAC: Authorizing `identity:get_project(project_id=5667e30274a94890a1493a76daa4eced)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:09.385247 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db19ff82-4ee2-497e-9ad2-78f03d9824b4 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:09.385247 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 157/313] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 08:32:09 2022] GET /identity/v3/projects/5667e30274a94890a1493a76daa4eced => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:09.386069 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db19ff82-4ee2-497e-9ad2-78f03d9824b4 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:09.416139 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93429edf-376c-438f-90e7-420ff3949b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:09.416781 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93429edf-376c-438f-90e7-420ff3949b99 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:09.417068 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93429edf-376c-438f-90e7-420ff3949b99 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:09.419881 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93429edf-376c-438f-90e7-420ff3949b99 None admin] PATH_INFO: `/v3/groups/bc0af4e23b84458dbca74018fb17d6d2` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:09.419881 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93429edf-376c-438f-90e7-420ff3949b99 None admin] RBAC: Authorizing `identity:get_group(group_id=bc0af4e23b84458dbca74018fb17d6d2)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:09.421136 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93429edf-376c-438f-90e7-420ff3949b99 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:09.423550 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 157/314] 10.222.0.30 () {56 vars in 1180 bytes} [Sat Oct 15 08:32:09 2022] GET /identity/v3/groups/bc0af4e23b84458dbca74018fb17d6d2 => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:09.424656 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64caa415-78e2-42d6-ab1e-b0c2bdea95a3 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:09.424938 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64caa415-78e2-42d6-ab1e-b0c2bdea95a3 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:09.425225 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64caa415-78e2-42d6-ab1e-b0c2bdea95a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:09.448038 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-64caa415-78e2-42d6-ab1e-b0c2bdea95a3 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:32:09.449166 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-64caa415-78e2-42d6-ab1e-b0c2bdea95a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:32:09.514715 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db19ff82-4ee2-497e-9ad2-78f03d9824b4 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:09.537408 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 158/315] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:32:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 08:32:09.538596 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64caa415-78e2-42d6-ab1e-b0c2bdea95a3 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:09.556955 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570d4c80-6d63-48ee-9ce7-74bdb80565c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:09.557570 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d4c80-6d63-48ee-9ce7-74bdb80565c0 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:09.557909 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d4c80-6d63-48ee-9ce7-74bdb80565c0 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:09.558184 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d4c80-6d63-48ee-9ce7-74bdb80565c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:09.589276 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:09.597448 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:09.637237 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570d4c80-6d63-48ee-9ce7-74bdb80565c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:09.638728 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570d4c80-6d63-48ee-9ce7-74bdb80565c0 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:09.667059 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64caa415-78e2-42d6-ab1e-b0c2bdea95a3 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:09.708227 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccda919a-c46d-45d6-91ea-ca0b77693da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:09.708841 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccda919a-c46d-45d6-91ea-ca0b77693da2 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:09.709124 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccda919a-c46d-45d6-91ea-ca0b77693da2 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:09.712401 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccda919a-c46d-45d6-91ea-ca0b77693da2 None admin] PATH_INFO: `/v3/groups/bc0af4e23b84458dbca74018fb17d6d2` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:09.713496 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccda919a-c46d-45d6-91ea-ca0b77693da2 None admin] RBAC: Authorizing `identity:get_group(group_id=bc0af4e23b84458dbca74018fb17d6d2)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:09.714993 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccda919a-c46d-45d6-91ea-ca0b77693da2 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:09.716756 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 159/316] 10.222.0.30 () {56 vars in 1180 bytes} [Sat Oct 15 08:32:09 2022] GET /identity/v3/groups/bc0af4e23b84458dbca74018fb17d6d2 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:09.721617 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccda919a-c46d-45d6-91ea-ca0b77693da2 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:09.804927 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 158/317] 10.222.0.30 () {58 vars in 1282 bytes} [Sat Oct 15 08:32:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 08:32:09.806873 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570d4c80-6d63-48ee-9ce7-74bdb80565c0 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:09.855992 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccda919a-c46d-45d6-91ea-ca0b77693da2 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:09.873633 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:09.887718 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:09.910120 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b74668-1532-4e3d-80d4-8dd3c682c695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:09.911455 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b74668-1532-4e3d-80d4-8dd3c682c695 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:09.911750 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b74668-1532-4e3d-80d4-8dd3c682c695 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:09.912035 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b74668-1532-4e3d-80d4-8dd3c682c695 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:09.912798 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b74668-1532-4e3d-80d4-8dd3c682c695 None admin] RBAC: Adding query filter params (group.id=bc0af4e23b84458dbca74018fb17d6d2, role.id=d236b691c1434103b73cc4fd0b39fec1, scope.project.id=5667e30274a94890a1493a76daa4eced) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:09.913112 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b74668-1532-4e3d-80d4-8dd3c682c695 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:09.915549 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b74668-1532-4e3d-80d4-8dd3c682c695 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:09.924256 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 160/318] 10.222.0.30 () {56 vars in 1376 bytes} [Sat Oct 15 08:32:09 2022] GET /identity/v3/role_assignments?group.id=bc0af4e23b84458dbca74018fb17d6d2&scope.project.id=5667e30274a94890a1493a76daa4eced&role.id=d236b691c1434103b73cc4fd0b39fec1 => generated 611 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:09.925088 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b74668-1532-4e3d-80d4-8dd3c682c695 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:09.945899 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570d4c80-6d63-48ee-9ce7-74bdb80565c0 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:09.989254 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0638ec59-7070-4683-a1b6-196c80d1499a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:09.990206 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0638ec59-7070-4683-a1b6-196c80d1499a None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:09.990669 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0638ec59-7070-4683-a1b6-196c80d1499a None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:09.991210 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0638ec59-7070-4683-a1b6-196c80d1499a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:09.992241 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0638ec59-7070-4683-a1b6-196c80d1499a None admin] RBAC: Adding query filter params (group.id=bc0af4e23b84458dbca74018fb17d6d2, role.id=3248f0268738489b883041851d681bb7, scope.project.id=5667e30274a94890a1493a76daa4eced) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:09.992732 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0638ec59-7070-4683-a1b6-196c80d1499a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:09.994992 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0638ec59-7070-4683-a1b6-196c80d1499a None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:10.008116 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 159/319] 10.222.0.30 () {56 vars in 1376 bytes} [Sat Oct 15 08:32:09 2022] GET /identity/v3/role_assignments?group.id=bc0af4e23b84458dbca74018fb17d6d2&scope.project.id=5667e30274a94890a1493a76daa4eced&role.id=3248f0268738489b883041851d681bb7 => generated 611 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:10.008961 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0638ec59-7070-4683-a1b6-196c80d1499a None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:10.064518 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b74668-1532-4e3d-80d4-8dd3c682c695 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:10.111525 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a36b21-b27c-41e0-aeaa-64375879fd23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:10.112176 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a36b21-b27c-41e0-aeaa-64375879fd23 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:10.112457 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a36b21-b27c-41e0-aeaa-64375879fd23 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:10.113027 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a36b21-b27c-41e0-aeaa-64375879fd23 None admin] PATH_INFO: `/v3/projects/7537ad5b7fef46308a69ce89f36209c2` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:10.114733 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a36b21-b27c-41e0-aeaa-64375879fd23 None admin] RBAC: Authorizing `identity:get_project(project_id=7537ad5b7fef46308a69ce89f36209c2)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:10.116182 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a36b21-b27c-41e0-aeaa-64375879fd23 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:10.118772 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 161/320] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 08:32:09 2022] GET /identity/v3/projects/7537ad5b7fef46308a69ce89f36209c2 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:10.120353 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23790526-0682-4379-a686-73487516ac32 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:10.120640 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23790526-0682-4379-a686-73487516ac32 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:10.120917 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23790526-0682-4379-a686-73487516ac32 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:10.122459 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 162/321] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:32:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:32:10.127514 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23790526-0682-4379-a686-73487516ac32 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:10.133642 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0638ec59-7070-4683-a1b6-196c80d1499a None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:10.161874 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:10.182697 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55012f1a-3f6e-4284-823e-4e65aa3c8572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:10.183449 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55012f1a-3f6e-4284-823e-4e65aa3c8572 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:10.183799 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55012f1a-3f6e-4284-823e-4e65aa3c8572 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:10.184144 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55012f1a-3f6e-4284-823e-4e65aa3c8572 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:10.201300 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:10.202849 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55012f1a-3f6e-4284-823e-4e65aa3c8572 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:32:10.212506 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-55012f1a-3f6e-4284-823e-4e65aa3c8572 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 08:32:10.212506 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:10.212506 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:10.212506 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:10.212506 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:10.212506 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:10.212506 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:10.212506 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:10.212506 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:10.212506 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:10.212506 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:10.212506 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:10.212506 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:32:10.212506 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:32:10.212506 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:32:10.212506 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:32:10.214755 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:32:10.214755 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:32:10.214755 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:32:10.214755 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:32:10.214755 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:10.214755 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:10.214755 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:10.214755 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:10.214755 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:10.214755 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:10.214755 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:10.214755 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:10.214755 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:10.214755 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:10.214755 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:10.214755 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:10.216688 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:10.216688 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:10.216688 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:10.216688 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:10.216688 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:32:10.216688 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:32:10.216688 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:32:10.216688 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:32:10.216688 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:32:10.216688 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:32:10.216688 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 08:32:10.216688 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:10.216688 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 160/322] 10.222.0.30 () {56 vars in 1114 bytes} [Sat Oct 15 08:32:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:10.216688 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeac413-73e0-4d86-9541-939fac6cc999 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:10.218440 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeac413-73e0-4d86-9541-939fac6cc999 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:10.218440 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeac413-73e0-4d86-9541-939fac6cc999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:10.240265 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-3aeac413-73e0-4d86-9541-939fac6cc999 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:32:10.241416 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-3aeac413-73e0-4d86-9541-939fac6cc999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:32:10.282446 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23790526-0682-4379-a686-73487516ac32 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:10.329060 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa4bd0a-4dd9-4f8c-b25c-a7cd234123e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:10.330138 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa4bd0a-4dd9-4f8c-b25c-a7cd234123e7 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:10.330598 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa4bd0a-4dd9-4f8c-b25c-a7cd234123e7 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:10.331010 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 161/323] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:32:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 08:32:10.331312 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa4bd0a-4dd9-4f8c-b25c-a7cd234123e7 None admin] PATH_INFO: `/v3/groups/bc0af4e23b84458dbca74018fb17d6d2` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:10.331857 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeac413-73e0-4d86-9541-939fac6cc999 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:10.332852 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa4bd0a-4dd9-4f8c-b25c-a7cd234123e7 None admin] RBAC: Authorizing `identity:get_group(group_id=bc0af4e23b84458dbca74018fb17d6d2)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:10.334609 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa4bd0a-4dd9-4f8c-b25c-a7cd234123e7 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:10.341145 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 163/324] 10.222.0.30 () {56 vars in 1180 bytes} [Sat Oct 15 08:32:10 2022] GET /identity/v3/groups/bc0af4e23b84458dbca74018fb17d6d2 => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:10.346830 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa4bd0a-4dd9-4f8c-b25c-a7cd234123e7 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:10.468672 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeac413-73e0-4d86-9541-939fac6cc999 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:10.480682 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa4bd0a-4dd9-4f8c-b25c-a7cd234123e7 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:10.485219 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:10.501908 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:10.517422 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c39f287-43ae-4ac3-aeb7-e31656c79d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:10.518213 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c39f287-43ae-4ac3-aeb7-e31656c79d52 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:10.518561 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c39f287-43ae-4ac3-aeb7-e31656c79d52 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:10.518837 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c39f287-43ae-4ac3-aeb7-e31656c79d52 None admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:10.519503 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c39f287-43ae-4ac3-aeb7-e31656c79d52 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:10.519905 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c39f287-43ae-4ac3-aeb7-e31656c79d52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:10.521289 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c39f287-43ae-4ac3-aeb7-e31656c79d52 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:10.528365 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ba0668-4959-4bd3-9714-c1544c6db8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:10.528975 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba0668-4959-4bd3-9714-c1544c6db8c1 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:10.529272 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba0668-4959-4bd3-9714-c1544c6db8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:10.529542 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba0668-4959-4bd3-9714-c1544c6db8c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:10.530302 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ba0668-4959-4bd3-9714-c1544c6db8c1 None admin] RBAC: Adding query filter params (group.id=bc0af4e23b84458dbca74018fb17d6d2, role.id=3248f0268738489b883041851d681bb7, scope.project.id=7537ad5b7fef46308a69ce89f36209c2) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:10.530618 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ba0668-4959-4bd3-9714-c1544c6db8c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:10.530772 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 162/325] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 08:32:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:10.532041 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ba0668-4959-4bd3-9714-c1544c6db8c1 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:10.536945 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c39f287-43ae-4ac3-aeb7-e31656c79d52 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:10.539352 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 164/326] 10.222.0.30 () {56 vars in 1376 bytes} [Sat Oct 15 08:32:10 2022] GET /identity/v3/role_assignments?group.id=bc0af4e23b84458dbca74018fb17d6d2&scope.project.id=7537ad5b7fef46308a69ce89f36209c2&role.id=3248f0268738489b883041851d681bb7 => generated 611 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:10.614058 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ba0668-4959-4bd3-9714-c1544c6db8c1 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:10.674947 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c39f287-43ae-4ac3-aeb7-e31656c79d52 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:10.720007 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ee60b4-3e53-42ac-9d4a-43b06826a501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:10.721996 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee60b4-3e53-42ac-9d4a-43b06826a501 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:10.721996 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee60b4-3e53-42ac-9d4a-43b06826a501 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:10.721996 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee60b4-3e53-42ac-9d4a-43b06826a501 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:10.758334 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ba0668-4959-4bd3-9714-c1544c6db8c1 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:10.782212 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:10.799666 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054af9bd-9b33-47e5-879f-15cecc389ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:10.800404 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054af9bd-9b33-47e5-879f-15cecc389ceb None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:10.800699 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054af9bd-9b33-47e5-879f-15cecc389ceb None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:10.800978 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054af9bd-9b33-47e5-879f-15cecc389ceb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:10.813874 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054af9bd-9b33-47e5-879f-15cecc389ceb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:32:10.816478 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ee60b4-3e53-42ac-9d4a-43b06826a501 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:10.818065 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ee60b4-3e53-42ac-9d4a-43b06826a501 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:10.819381 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-054af9bd-9b33-47e5-879f-15cecc389ceb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 08:32:10.819381 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:10.819381 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:10.819381 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:10.819381 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:10.819381 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:10.819381 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:10.819381 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:10.819381 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:10.819381 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:10.819381 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:10.819381 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:10.819381 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:32:10.819381 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:32:10.819381 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:32:10.819381 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:32:10.820292 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:32:10.820292 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:32:10.820292 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:32:10.820292 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:32:10.820292 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:10.820292 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:10.820292 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:10.820292 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:10.820292 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:10.820292 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:10.820292 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:10.820292 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:10.820292 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:10.820292 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:10.820292 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:10.820292 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:10.821076 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:10.821076 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:10.821076 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:10.821076 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:10.821076 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:32:10.821076 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:32:10.821076 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:32:10.821076 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:32:10.821076 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:32:10.821076 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:32:10.821076 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 08:32:10.821076 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:10.821076 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 165/327] 10.222.0.30 () {56 vars in 1096 bytes} [Sat Oct 15 08:32:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:10.827661 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054af9bd-9b33-47e5-879f-15cecc389ceb None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:10.873852 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:10.939262 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 163/328] 10.222.0.30 () {58 vars in 1282 bytes} [Sat Oct 15 08:32:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 08:32:10.946296 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ee60b4-3e53-42ac-9d4a-43b06826a501 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:10.957506 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054af9bd-9b33-47e5-879f-15cecc389ceb None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:10.999175 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b51efd-7054-45cc-9883-862c834ba84d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:11.001230 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b51efd-7054-45cc-9883-862c834ba84d None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:11.001230 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b51efd-7054-45cc-9883-862c834ba84d None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:11.001230 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b51efd-7054-45cc-9883-862c834ba84d None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:11.001230 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b51efd-7054-45cc-9883-862c834ba84d None admin] RBAC: Adding query filter params (name=admin) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:11.001540 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b51efd-7054-45cc-9883-862c834ba84d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:11.002703 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b51efd-7054-45cc-9883-862c834ba84d None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:11.013309 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 166/329] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 08:32:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 08:32:11.018819 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b51efd-7054-45cc-9883-862c834ba84d None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:11.043946 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:11.074990 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ee60b4-3e53-42ac-9d4a-43b06826a501 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:11.118475 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b56385-a9ad-426e-b458-7d79beeb5f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:11.119095 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b56385-a9ad-426e-b458-7d79beeb5f69 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:11.119479 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b56385-a9ad-426e-b458-7d79beeb5f69 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:11.119756 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b56385-a9ad-426e-b458-7d79beeb5f69 None admin] PATH_INFO: `/v3/projects/7537ad5b7fef46308a69ce89f36209c2` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:11.120974 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b56385-a9ad-426e-b458-7d79beeb5f69 None admin] RBAC: Authorizing `identity:get_project(project_id=7537ad5b7fef46308a69ce89f36209c2)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:11.122545 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b56385-a9ad-426e-b458-7d79beeb5f69 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:11.125465 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 164/330] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 08:32:10 2022] GET /identity/v3/projects/7537ad5b7fef46308a69ce89f36209c2 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:11.130114 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:11.130765 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b56385-a9ad-426e-b458-7d79beeb5f69 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:11.133165 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b51efd-7054-45cc-9883-862c834ba84d None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:11.173705 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fae529-c503-4785-aa32-b43b0d09a032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:11.174341 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fae529-c503-4785-aa32-b43b0d09a032 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:11.174630 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fae529-c503-4785-aa32-b43b0d09a032 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:11.174911 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fae529-c503-4785-aa32-b43b0d09a032 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:11.238781 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b56385-a9ad-426e-b458-7d79beeb5f69 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:11.242240 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fae529-c503-4785-aa32-b43b0d09a032 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:11.244194 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fae529-c503-4785-aa32-b43b0d09a032 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:11.273489 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf34268-f7c3-4086-8b66-e19c20145b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:11.274078 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf34268-f7c3-4086-8b66-e19c20145b49 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:11.274366 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf34268-f7c3-4086-8b66-e19c20145b49 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:11.274666 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf34268-f7c3-4086-8b66-e19c20145b49 None admin] PATH_INFO: `/v3/groups/bc0af4e23b84458dbca74018fb17d6d2` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:11.275766 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf34268-f7c3-4086-8b66-e19c20145b49 None admin] RBAC: Authorizing `identity:get_group(group_id=bc0af4e23b84458dbca74018fb17d6d2)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:11.277203 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf34268-f7c3-4086-8b66-e19c20145b49 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:11.278809 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 165/331] 10.222.0.30 () {56 vars in 1180 bytes} [Sat Oct 15 08:32:11 2022] GET /identity/v3/groups/bc0af4e23b84458dbca74018fb17d6d2 => generated 226 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 08:32:11.284540 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf34268-f7c3-4086-8b66-e19c20145b49 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:11.295919 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:11.338086 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:11.346402 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 167/332] 10.222.0.30 () {58 vars in 1282 bytes} [Sat Oct 15 08:32:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 08:32:11.347419 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fae529-c503-4785-aa32-b43b0d09a032 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:11.383840 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf34268-f7c3-4086-8b66-e19c20145b49 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:11.418732 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687721ca-54d2-4bb1-bdc7-18a1704f488b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:11.419344 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687721ca-54d2-4bb1-bdc7-18a1704f488b None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:11.419627 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687721ca-54d2-4bb1-bdc7-18a1704f488b None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:11.419904 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687721ca-54d2-4bb1-bdc7-18a1704f488b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:11.420625 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687721ca-54d2-4bb1-bdc7-18a1704f488b None admin] RBAC: Adding query filter params (group.id=bc0af4e23b84458dbca74018fb17d6d2, role.id=d236b691c1434103b73cc4fd0b39fec1, scope.project.id=7537ad5b7fef46308a69ce89f36209c2) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:11.420934 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687721ca-54d2-4bb1-bdc7-18a1704f488b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:11.422409 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687721ca-54d2-4bb1-bdc7-18a1704f488b None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:11.428224 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 166/333] 10.222.0.30 () {56 vars in 1376 bytes} [Sat Oct 15 08:32:11 2022] GET /identity/v3/role_assignments?group.id=bc0af4e23b84458dbca74018fb17d6d2&scope.project.id=7537ad5b7fef46308a69ce89f36209c2&role.id=d236b691c1434103b73cc4fd0b39fec1 => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 08:32:11.443750 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fae529-c503-4785-aa32-b43b0d09a032 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:11.480602 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57915f3a-f892-44d9-906a-c9eda6a29d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:11.481252 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57915f3a-f892-44d9-906a-c9eda6a29d26 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:11.481531 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57915f3a-f892-44d9-906a-c9eda6a29d26 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:11.481838 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57915f3a-f892-44d9-906a-c9eda6a29d26 None admin] PATH_INFO: `/v3/projects/d200ced7ea7746bab37279a9888da771` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:11.483031 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57915f3a-f892-44d9-906a-c9eda6a29d26 None admin] RBAC: Authorizing `identity:get_project(project_id=d200ced7ea7746bab37279a9888da771)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:11.484486 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57915f3a-f892-44d9-906a-c9eda6a29d26 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:11.486445 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 168/334] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 08:32:11 2022] GET /identity/v3/projects/d200ced7ea7746bab37279a9888da771 => generated 344 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:11.491954 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687721ca-54d2-4bb1-bdc7-18a1704f488b None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:11.573210 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:11.600770 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687721ca-54d2-4bb1-bdc7-18a1704f488b None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:11.635752 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50b8617-6c31-4e81-8397-0b89e4e32460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:11.636387 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50b8617-6c31-4e81-8397-0b89e4e32460 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:11.636668 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50b8617-6c31-4e81-8397-0b89e4e32460 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:11.636947 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50b8617-6c31-4e81-8397-0b89e4e32460 None admin] PATH_INFO: `/v3/groups/209444e529574ef1a5d5d1a0075cb242` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:11.638069 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50b8617-6c31-4e81-8397-0b89e4e32460 None admin] RBAC: Authorizing `identity:get_group(group_id=209444e529574ef1a5d5d1a0075cb242)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:11.639497 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50b8617-6c31-4e81-8397-0b89e4e32460 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:11.641087 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 167/335] 10.222.0.30 () {56 vars in 1180 bytes} [Sat Oct 15 08:32:11 2022] GET /identity/v3/groups/209444e529574ef1a5d5d1a0075cb242 => generated 229 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:11.647187 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57915f3a-f892-44d9-906a-c9eda6a29d26 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:11.716386 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:11.749873 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57915f3a-f892-44d9-906a-c9eda6a29d26 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:11.785165 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925560a8-6c7c-43d8-bcfa-fce7cdfaeb3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:11.785817 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925560a8-6c7c-43d8-bcfa-fce7cdfaeb3a None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:11.786353 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925560a8-6c7c-43d8-bcfa-fce7cdfaeb3a None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:11.786644 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925560a8-6c7c-43d8-bcfa-fce7cdfaeb3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:11.787368 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925560a8-6c7c-43d8-bcfa-fce7cdfaeb3a None admin] RBAC: Adding query filter params (group.id=209444e529574ef1a5d5d1a0075cb242, role.id=14615e30ed3f4d8cb78ee5e0bf9b5761, scope.project.id=d200ced7ea7746bab37279a9888da771) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:11.787679 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925560a8-6c7c-43d8-bcfa-fce7cdfaeb3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:11.789100 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925560a8-6c7c-43d8-bcfa-fce7cdfaeb3a None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:11.795442 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 169/336] 10.222.0.30 () {56 vars in 1376 bytes} [Sat Oct 15 08:32:11 2022] GET /identity/v3/role_assignments?group.id=209444e529574ef1a5d5d1a0075cb242&scope.project.id=d200ced7ea7746bab37279a9888da771&role.id=14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:14.266057 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862e6c5b-cca7-4071-a8a0-dd521f245d49 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:14.266890 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862e6c5b-cca7-4071-a8a0-dd521f245d49 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:14.266890 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862e6c5b-cca7-4071-a8a0-dd521f245d49 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:14.268449 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 168/337] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:32:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:32:14.275158 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5662ca6e-5683-4458-b4d4-6aad0d910ea7 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:14.276035 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5662ca6e-5683-4458-b4d4-6aad0d910ea7 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:14.276530 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5662ca6e-5683-4458-b4d4-6aad0d910ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:14.306380 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-5662ca6e-5683-4458-b4d4-6aad0d910ea7 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:32:14.307910 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-5662ca6e-5683-4458-b4d4-6aad0d910ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:32:14.396179 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 170/338] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:32:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 08:32:14.546569 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd195365-ab69-4d5c-8f45-16ebde2f91e0 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:14.547187 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd195365-ab69-4d5c-8f45-16ebde2f91e0 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:14.547688 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd195365-ab69-4d5c-8f45-16ebde2f91e0 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:14.549544 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 169/339] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:32:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:32:14.556244 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8bc858-4f72-4dfc-802e-c7657eae9d72 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:14.556814 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8bc858-4f72-4dfc-802e-c7657eae9d72 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:14.557299 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8bc858-4f72-4dfc-802e-c7657eae9d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:14.586659 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-0a8bc858-4f72-4dfc-802e-c7657eae9d72 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:32:14.588311 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-0a8bc858-4f72-4dfc-802e-c7657eae9d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:32:14.657132 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:14.670295 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 171/340] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:32:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 08:32:14.870035 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd195365-ab69-4d5c-8f45-16ebde2f91e0 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:15.001185 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd195365-ab69-4d5c-8f45-16ebde2f91e0 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:15.001830 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e782f-6369-4b34-99d4-5be7f5fa939c None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:15.002414 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e782f-6369-4b34-99d4-5be7f5fa939c None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:15.002895 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e782f-6369-4b34-99d4-5be7f5fa939c None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:15.004521 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 172/341] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:32:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:32:15.010011 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306348ce-deb2-4b56-91de-886e1fb2303d None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:15.010295 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306348ce-deb2-4b56-91de-886e1fb2303d None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:15.010571 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306348ce-deb2-4b56-91de-886e1fb2303d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:15.033170 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-306348ce-deb2-4b56-91de-886e1fb2303d None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:32:15.034401 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-306348ce-deb2-4b56-91de-886e1fb2303d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:32:15.044898 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571cf7ac-4b93-4b59-8ef4-8d9c5bebae09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:15.045533 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571cf7ac-4b93-4b59-8ef4-8d9c5bebae09 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:15.045939 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571cf7ac-4b93-4b59-8ef4-8d9c5bebae09 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:15.046160 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571cf7ac-4b93-4b59-8ef4-8d9c5bebae09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:15.057431 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571cf7ac-4b93-4b59-8ef4-8d9c5bebae09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:15.058932 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571cf7ac-4b93-4b59-8ef4-8d9c5bebae09 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:15.060968 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:15.076306 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-571cf7ac-4b93-4b59-8ef4-8d9c5bebae09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:15.076306 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:15.076306 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:15.076306 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:15.076306 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:15.076306 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:15.076306 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:15.076306 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:15.076306 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:15.076306 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:15.076306 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:15.076306 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:15.076306 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 08:32:15.076306 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 08:32:15.076306 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 08:32:15.076306 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 08:32:15.077486 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:15.077486 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:15.077486 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:15.077486 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:15.077486 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:15.077486 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:15.077486 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:15.077486 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:15.077486 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:15.077486 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:15.077486 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:15.077486 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:15.077486 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:15.077486 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:15.077486 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:15.077486 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:15.077486 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 08:32:15.078617 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 08:32:15.078617 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:15.078617 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:15.080442 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 170/342] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:32:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:15.081810 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b11be-bbc0-46ad-98a8-3e5d8123d713 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:15.082100 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b11be-bbc0-46ad-98a8-3e5d8123d713 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:15.082379 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b11be-bbc0-46ad-98a8-3e5d8123d713 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:15.083794 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 171/343] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:32:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:32:15.084316 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417b11be-bbc0-46ad-98a8-3e5d8123d713 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:15.112604 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 173/344] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:32:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 08:32:15.113413 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306348ce-deb2-4b56-91de-886e1fb2303d None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:15.221829 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417b11be-bbc0-46ad-98a8-3e5d8123d713 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:15.222677 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306348ce-deb2-4b56-91de-886e1fb2303d None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:15.249269 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:15.263258 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed43405-7d8e-4f29-a685-f24dad1a5ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:15.263886 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed43405-7d8e-4f29-a685-f24dad1a5ced None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:15.264169 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed43405-7d8e-4f29-a685-f24dad1a5ced None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:15.264447 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed43405-7d8e-4f29-a685-f24dad1a5ced None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:15.265377 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed43405-7d8e-4f29-a685-f24dad1a5ced None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:15.265704 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed43405-7d8e-4f29-a685-f24dad1a5ced None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:15.267395 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed43405-7d8e-4f29-a685-f24dad1a5ced None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:15.273826 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0b4398-4853-4971-8ad6-5fb033c6497d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:15.276146 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b4398-4853-4971-8ad6-5fb033c6497d None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:15.276649 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b4398-4853-4971-8ad6-5fb033c6497d None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:15.277717 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b4398-4853-4971-8ad6-5fb033c6497d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:15.280499 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 174/345] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 08:32:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:15.282581 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1925fe-33b7-4832-8bea-ea1c1f8d6055 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:15.283131 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1925fe-33b7-4832-8bea-ea1c1f8d6055 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:15.283562 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1925fe-33b7-4832-8bea-ea1c1f8d6055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:15.290630 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0b4398-4853-4971-8ad6-5fb033c6497d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:15.293443 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0b4398-4853-4971-8ad6-5fb033c6497d None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:15.303189 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-ea1925fe-33b7-4832-8bea-ea1c1f8d6055 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:32:15.304607 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-ea1925fe-33b7-4832-8bea-ea1c1f8d6055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:32:15.311548 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-3c0b4398-4853-4971-8ad6-5fb033c6497d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:15.311548 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:15.311548 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:15.311548 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:15.311548 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:15.311548 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:15.311548 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:15.311548 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:15.311548 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:15.311548 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:15.311548 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:15.311548 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:15.311548 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 08:32:15.311548 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 08:32:15.311548 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 08:32:15.311548 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 08:32:15.312450 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:15.312450 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:15.312450 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:15.312450 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:15.312450 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:15.312450 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:15.312450 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:15.312450 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:15.312450 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:15.312450 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:15.312450 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:15.312450 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:15.312450 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:15.312450 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:15.312450 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:15.312450 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:15.312450 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 08:32:15.313310 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 08:32:15.313310 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:15.313310 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:15.317756 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 172/346] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:32:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:15.320387 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0b4398-4853-4971-8ad6-5fb033c6497d None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:15.385432 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:15.399591 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 175/347] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:32:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 08:32:15.400627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1925fe-33b7-4832-8bea-ea1c1f8d6055 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:15.452902 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0b4398-4853-4971-8ad6-5fb033c6497d None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:15.498966 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbfc95d-fdfe-4324-b981-f17e79a99c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:15.499946 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbfc95d-fdfe-4324-b981-f17e79a99c79 None admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:15.500235 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbfc95d-fdfe-4324-b981-f17e79a99c79 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:15.500509 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbfc95d-fdfe-4324-b981-f17e79a99c79 None admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:15.501150 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbfc95d-fdfe-4324-b981-f17e79a99c79 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:15.503145 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbfc95d-fdfe-4324-b981-f17e79a99c79 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:15.514958 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1925fe-33b7-4832-8bea-ea1c1f8d6055 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:15.531751 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:15.537585 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 173/348] 10.222.0.30 () {60 vars in 1132 bytes} [Sat Oct 15 08:32:15 2022] POST /identity/v3/users => generated 246 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:32:15.538467 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbfc95d-fdfe-4324-b981-f17e79a99c79 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:15.558862 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b048b41-3d31-488a-81f3-7404fc720eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:15.559657 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b048b41-3d31-488a-81f3-7404fc720eb8 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:15.559948 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b048b41-3d31-488a-81f3-7404fc720eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:15.560657 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b048b41-3d31-488a-81f3-7404fc720eb8 None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:15.561630 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b048b41-3d31-488a-81f3-7404fc720eb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:15.561982 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b048b41-3d31-488a-81f3-7404fc720eb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:15.563905 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b048b41-3d31-488a-81f3-7404fc720eb8 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:15.580124 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 176/349] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 08:32:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:15.592206 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b048b41-3d31-488a-81f3-7404fc720eb8 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:15.722552 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbfc95d-fdfe-4324-b981-f17e79a99c79 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:15.724237 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:15.765745 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b048b41-3d31-488a-81f3-7404fc720eb8 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:15.775194 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68a15e8-99ce-4cd7-836c-b97c30386c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:15.775483 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a15e8-99ce-4cd7-836c-b97c30386c43 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:15.776548 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a15e8-99ce-4cd7-836c-b97c30386c43 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:15.776548 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a15e8-99ce-4cd7-836c-b97c30386c43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:15.789159 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68a15e8-99ce-4cd7-836c-b97c30386c43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:15.790735 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68a15e8-99ce-4cd7-836c-b97c30386c43 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:15.807281 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-f68a15e8-99ce-4cd7-836c-b97c30386c43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:15.807281 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:15.807281 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:15.807281 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:15.807281 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:15.807281 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:15.807281 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:15.807281 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:15.807281 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:15.807281 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:15.807281 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:15.807281 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:15.807281 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 08:32:15.807281 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 08:32:15.807281 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 08:32:15.807281 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 08:32:15.808203 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:15.808203 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:15.808203 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:15.808203 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:15.808203 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:15.808203 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:15.808203 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:15.808203 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:15.808203 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:15.808203 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:15.808203 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:15.808203 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:15.808203 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:15.808203 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:15.808203 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:15.808203 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:15.808203 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 08:32:15.809023 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 08:32:15.809023 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:15.809023 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:15.816171 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 174/350] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:32:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:15.816171 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68a15e8-99ce-4cd7-836c-b97c30386c43 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:15.818166 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdeb34a-2e9a-4a44-916b-54b7eb650312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:15.818790 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdeb34a-2e9a-4a44-916b-54b7eb650312 None admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:15.819074 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdeb34a-2e9a-4a44-916b-54b7eb650312 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:15.819350 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdeb34a-2e9a-4a44-916b-54b7eb650312 None admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:15.820118 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdeb34a-2e9a-4a44-916b-54b7eb650312 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:15.821920 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdeb34a-2e9a-4a44-916b-54b7eb650312 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:15.853903 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 177/351] 10.222.0.30 () {60 vars in 1132 bytes} [Sat Oct 15 08:32:15 2022] POST /identity/v3/users => generated 248 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 08:32:15.856273 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdeb34a-2e9a-4a44-916b-54b7eb650312 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:15.920477 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:16.018008 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68a15e8-99ce-4cd7-836c-b97c30386c43 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:16.073940 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdeb34a-2e9a-4a44-916b-54b7eb650312 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:16.080950 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d2ab23-34cd-400a-a09a-1d5b1cb44c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:16.081612 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d2ab23-34cd-400a-a09a-1d5b1cb44c48 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:16.081941 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d2ab23-34cd-400a-a09a-1d5b1cb44c48 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:16.082199 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d2ab23-34cd-400a-a09a-1d5b1cb44c48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:16.094905 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d2ab23-34cd-400a-a09a-1d5b1cb44c48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:16.096493 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d2ab23-34cd-400a-a09a-1d5b1cb44c48 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:16.114737 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e9c556-3632-4e71-b0dd-73b27e65ea36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:16.115072 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-24d2ab23-34cd-400a-a09a-1d5b1cb44c48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:16.115072 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:16.115072 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:16.115072 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:16.115072 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:16.115072 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:16.115072 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:16.115072 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:16.115072 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:16.115072 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:16.115072 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:16.115072 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:16.115072 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 08:32:16.115072 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 08:32:16.115072 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 08:32:16.115072 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 08:32:16.115912 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:16.115912 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:16.115912 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:16.115912 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:16.115912 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:16.115912 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:16.115912 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:16.115912 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:16.115912 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:16.115912 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:16.115912 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:16.115912 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:16.115912 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:16.115912 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:16.115912 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:16.115912 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:16.115912 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 08:32:16.116777 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 08:32:16.116777 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:16.116777 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:16.118782 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9c556-3632-4e71-b0dd-73b27e65ea36 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:16.119304 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9c556-3632-4e71-b0dd-73b27e65ea36 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:16.119801 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9c556-3632-4e71-b0dd-73b27e65ea36 None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:16.120696 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e9c556-3632-4e71-b0dd-73b27e65ea36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:16.121193 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e9c556-3632-4e71-b0dd-73b27e65ea36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:16.121302 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 175/352] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:32:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:16.122513 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d2ab23-34cd-400a-a09a-1d5b1cb44c48 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:16.123484 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e9c556-3632-4e71-b0dd-73b27e65ea36 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:16.133646 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:16.139066 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 178/353] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 08:32:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:16.144852 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e9c556-3632-4e71-b0dd-73b27e65ea36 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:16.245386 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:16.259512 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e9c556-3632-4e71-b0dd-73b27e65ea36 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:16.261215 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d2ab23-34cd-400a-a09a-1d5b1cb44c48 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:16.296907 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88db8ce-c1f1-4191-acec-e7ff54c0b4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:16.298192 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88db8ce-c1f1-4191-acec-e7ff54c0b4bd None admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:16.298670 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88db8ce-c1f1-4191-acec-e7ff54c0b4bd None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:16.299137 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88db8ce-c1f1-4191-acec-e7ff54c0b4bd None admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:16.300027 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88db8ce-c1f1-4191-acec-e7ff54c0b4bd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:16.301796 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88db8ce-c1f1-4191-acec-e7ff54c0b4bd None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:16.304563 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca4eaba-d7c6-4fe9-9cb6-1b95c366a2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:16.305176 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca4eaba-d7c6-4fe9-9cb6-1b95c366a2ed None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:16.305457 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca4eaba-d7c6-4fe9-9cb6-1b95c366a2ed None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:16.305978 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca4eaba-d7c6-4fe9-9cb6-1b95c366a2ed None admin] PATH_INFO: `/v3/domains` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:16.306653 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca4eaba-d7c6-4fe9-9cb6-1b95c366a2ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:16.306958 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca4eaba-d7c6-4fe9-9cb6-1b95c366a2ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:16.308323 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca4eaba-d7c6-4fe9-9cb6-1b95c366a2ed None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:16.324649 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 176/354] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 08:32:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 08:32:16.326070 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7dc5e-4d1d-4dc0-8244-4187d28d6aa2 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:16.326401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7dc5e-4d1d-4dc0-8244-4187d28d6aa2 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:16.326678 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7dc5e-4d1d-4dc0-8244-4187d28d6aa2 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:16.333757 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 177/356] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:32:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:32:16.333757 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a7dc5e-4d1d-4dc0-8244-4187d28d6aa2 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:16.334176 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 179/356] 10.222.0.30 () {60 vars in 1132 bytes} [Sat Oct 15 08:32:16 2022] POST /identity/v3/users => generated 248 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:32:16.335949 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa596ab-85a8-490f-818a-8a4436116b5d None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:16.336330 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa596ab-85a8-490f-818a-8a4436116b5d None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:16.336703 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa596ab-85a8-490f-818a-8a4436116b5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:16.373036 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-bfa596ab-85a8-490f-818a-8a4436116b5d None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:32:16.374523 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-bfa596ab-85a8-490f-818a-8a4436116b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:32:16.438461 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:16.483130 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a7dc5e-4d1d-4dc0-8244-4187d28d6aa2 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:16.557617 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5dea16-9def-44fd-96a7-2399032a83a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:16.558361 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 180/357] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:32:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 08:32:16.560752 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5dea16-9def-44fd-96a7-2399032a83a0 None admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:16.561359 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5dea16-9def-44fd-96a7-2399032a83a0 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:16.561885 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5dea16-9def-44fd-96a7-2399032a83a0 None admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:16.562756 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5dea16-9def-44fd-96a7-2399032a83a0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:16.564439 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5dea16-9def-44fd-96a7-2399032a83a0 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:16.588180 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 178/358] 10.222.0.30 () {60 vars in 1132 bytes} [Sat Oct 15 08:32:16 2022] POST /identity/v3/users => generated 249 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:32:16.835911 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa596ab-85a8-490f-818a-8a4436116b5d None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:16.916258 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:16.944629 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa596ab-85a8-490f-818a-8a4436116b5d None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:16.987198 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04822186-dbf4-4abf-afaa-eb5a6fc08f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:16.987826 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04822186-dbf4-4abf-afaa-eb5a6fc08f37 None admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:16.988109 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04822186-dbf4-4abf-afaa-eb5a6fc08f37 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:16.988386 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04822186-dbf4-4abf-afaa-eb5a6fc08f37 None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:16.989084 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04822186-dbf4-4abf-afaa-eb5a6fc08f37 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:16.990500 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04822186-dbf4-4abf-afaa-eb5a6fc08f37 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:17.007430 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.sql.core [None req-04822186-dbf4-4abf-afaa-eb5a6fc08f37 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 15 08:32:17.007430 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 15 08:32:17.007430 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [parameters: {'id': 'fc9df68b15e140e3b4a58763ddb48bf8', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 15 08:32:17.007430 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88059) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 15 08:32:17.014663 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-04822186-dbf4-4abf-afaa-eb5a6fc08f37 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 15 08:32:17.014663 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:17.014663 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 15 08:32:17.014663 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 15 08:32:17.014663 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 15 08:32:17.014663 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 15 08:32:17.014663 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 15 08:32:17.014663 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application result = self._query(query) Oct 15 08:32:17.014663 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 15 08:32:17.014663 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application conn.query(q) Oct 15 08:32:17.014663 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 15 08:32:17.014663 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 15 08:32:17.014663 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 15 08:32:17.014663 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application result.read() Oct 15 08:32:17.014663 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 15 08:32:17.015539 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 15 08:32:17.015539 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 15 08:32:17.015539 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 15 08:32:17.015539 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 15 08:32:17.015539 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 15 08:32:17.015539 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 15 08:32:17.015539 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 15 08:32:17.015539 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 15 08:32:17.015539 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:17.015539 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 15 08:32:17.015539 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:17.015539 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:17.015539 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 15 08:32:17.015539 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 15 08:32:17.015539 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 15 08:32:17.015539 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return ref.to_dict() Oct 15 08:32:17.015539 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 15 08:32:17.015539 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application next(self.gen) Oct 15 08:32:17.017119 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 15 08:32:17.017119 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application yield resource Oct 15 08:32:17.017119 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 15 08:32:17.017119 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application next(self.gen) Oct 15 08:32:17.017119 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 15 08:32:17.017119 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self.session.rollback() Oct 15 08:32:17.017119 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 08:32:17.017119 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self.force_reraise() Oct 15 08:32:17.017119 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 08:32:17.017119 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise self.value Oct 15 08:32:17.017119 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 15 08:32:17.017119 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 15 08:32:17.017119 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 15 08:32:17.017119 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application session.commit() Oct 15 08:32:17.017119 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 15 08:32:17.017119 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 15 08:32:17.017119 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 15 08:32:17.018220 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._prepare_impl() Oct 15 08:32:17.018220 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 15 08:32:17.018220 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self.session.flush() Oct 15 08:32:17.018220 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 15 08:32:17.018220 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._flush(objects) Oct 15 08:32:17.018220 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 15 08:32:17.018220 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 15 08:32:17.018220 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 15 08:32:17.018220 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application compat.raise_( Oct 15 08:32:17.018220 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 15 08:32:17.018220 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception Oct 15 08:32:17.018220 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 15 08:32:17.018220 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application flush_context.execute() Oct 15 08:32:17.018220 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 15 08:32:17.018220 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rec.execute(self) Oct 15 08:32:17.018220 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 15 08:32:17.018220 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 15 08:32:17.019096 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 15 08:32:17.019096 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 15 08:32:17.019096 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 15 08:32:17.019096 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 15 08:32:17.019096 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 15 08:32:17.019096 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 15 08:32:17.019096 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 15 08:32:17.019096 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 15 08:32:17.019096 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 15 08:32:17.019096 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 15 08:32:17.019096 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 15 08:32:17.019096 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 15 08:32:17.019096 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 15 08:32:17.019096 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 15 08:32:17.019096 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 15 08:32:17.019096 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception Oct 15 08:32:17.019888 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 15 08:32:17.019888 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 15 08:32:17.019888 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 15 08:32:17.019888 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 15 08:32:17.019888 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 15 08:32:17.019888 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application result = self._query(query) Oct 15 08:32:17.019888 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 15 08:32:17.019888 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application conn.query(q) Oct 15 08:32:17.019888 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 15 08:32:17.019888 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 15 08:32:17.019888 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 15 08:32:17.019888 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application result.read() Oct 15 08:32:17.019888 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 15 08:32:17.019888 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 15 08:32:17.019888 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 15 08:32:17.019888 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 15 08:32:17.022797 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 15 08:32:17.022797 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 15 08:32:17.022797 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 15 08:32:17.022797 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 15 08:32:17.022797 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 15 08:32:17.022797 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 15 08:32:17.022797 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [parameters: {'id': 'fc9df68b15e140e3b4a58763ddb48bf8', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 15 08:32:17.022797 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 15 08:32:17.022797 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:17.022797 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 08:32:17.022797 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:17.022797 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:17.022797 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:17.022797 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:17.022797 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:17.022797 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:17.027800 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:17.027800 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:17.027800 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:17.027800 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:17.027800 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:17.027800 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:17.027800 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 15 08:32:17.027800 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 15 08:32:17.027800 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:17.027800 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:17.027800 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 15 08:32:17.027800 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 15 08:32:17.027800 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 15 08:32:17.027800 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 15 08:32:17.027800 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 15 08:32:17.027800 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:17.029874 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 181/359] 10.222.0.30 () {60 vars in 1131 bytes} [Sat Oct 15 08:32:16 2022] POST /identity/v3/roles => generated 144 bytes in 181 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:17.030013 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5dea16-9def-44fd-96a7-2399032a83a0 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:17.176674 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5dea16-9def-44fd-96a7-2399032a83a0 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:17.204308 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:17.216501 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ca9398-fe60-4db3-9846-b951d98949b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:17.217453 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ca9398-fe60-4db3-9846-b951d98949b6 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:17.222049 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ca9398-fe60-4db3-9846-b951d98949b6 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:17.222329 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ca9398-fe60-4db3-9846-b951d98949b6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:17.233166 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ca9398-fe60-4db3-9846-b951d98949b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:32:17.238506 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-10ca9398-fe60-4db3-9846-b951d98949b6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 08:32:17.238506 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:17.238506 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:17.238506 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:17.238506 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:17.238506 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:17.238506 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:17.238506 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:17.238506 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:17.238506 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:17.238506 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:17.238506 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:17.238506 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:32:17.238506 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:32:17.238506 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:32:17.238506 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:32:17.239407 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:32:17.239407 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:32:17.239407 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:32:17.239407 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:32:17.239407 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:17.239407 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:17.239407 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:17.239407 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:17.239407 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:17.239407 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:17.239407 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:17.239407 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:17.239407 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:17.239407 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:17.239407 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:17.239407 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:17.240861 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:17.240861 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:17.240861 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:17.240861 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:17.240861 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:32:17.240861 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:32:17.240861 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:32:17.240861 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:32:17.240861 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:32:17.240861 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:32:17.240861 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 08:32:17.240861 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:17.240861 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 179/360] 10.222.0.30 () {56 vars in 1114 bytes} [Sat Oct 15 08:32:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:17.246294 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04822186-dbf4-4abf-afaa-eb5a6fc08f37 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:17.378626 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04822186-dbf4-4abf-afaa-eb5a6fc08f37 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:17.426381 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd20939e-d456-4f41-84fe-46a6688d2f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:17.427063 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd20939e-d456-4f41-84fe-46a6688d2f82 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:17.427293 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd20939e-d456-4f41-84fe-46a6688d2f82 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:17.427573 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd20939e-d456-4f41-84fe-46a6688d2f82 None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:17.428253 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd20939e-d456-4f41-84fe-46a6688d2f82 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:17.428561 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd20939e-d456-4f41-84fe-46a6688d2f82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:17.429962 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd20939e-d456-4f41-84fe-46a6688d2f82 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:17.436104 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:17.439780 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 182/361] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 08:32:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:18.155374 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16de5bf-f149-4c03-b4c7-ac3b79ee24cf None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:18.155921 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16de5bf-f149-4c03-b4c7-ac3b79ee24cf None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:18.155921 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16de5bf-f149-4c03-b4c7-ac3b79ee24cf None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:18.157246 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 180/362] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:32:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:32:18.162485 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a38c47e-2050-4524-a374-fb65e92a3445 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:18.162775 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a38c47e-2050-4524-a374-fb65e92a3445 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:18.163054 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a38c47e-2050-4524-a374-fb65e92a3445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:18.181958 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-3a38c47e-2050-4524-a374-fb65e92a3445 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:32:18.183139 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-3a38c47e-2050-4524-a374-fb65e92a3445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:32:18.273189 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 183/363] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:32:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 08:32:18.449878 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e56d5-c5c6-4076-ad77-79a8364d548a None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:18.450078 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e56d5-c5c6-4076-ad77-79a8364d548a None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:18.450567 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e56d5-c5c6-4076-ad77-79a8364d548a None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:18.452071 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 181/364] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:32:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:32:18.457628 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f73c24a-7c3a-4351-b409-ba0dbc9b4afc None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:18.458561 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f73c24a-7c3a-4351-b409-ba0dbc9b4afc None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:18.459031 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f73c24a-7c3a-4351-b409-ba0dbc9b4afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:18.482092 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-9f73c24a-7c3a-4351-b409-ba0dbc9b4afc None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:32:18.483531 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-9f73c24a-7c3a-4351-b409-ba0dbc9b4afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:32:18.580207 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 184/365] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:32:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 08:32:18.614698 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1e56d5-c5c6-4076-ad77-79a8364d548a None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:18.741550 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1e56d5-c5c6-4076-ad77-79a8364d548a None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:18.785134 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8f633c-030a-47b0-ae1d-16dfe684e725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:18.785800 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f633c-030a-47b0-ae1d-16dfe684e725 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:18.786087 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f633c-030a-47b0-ae1d-16dfe684e725 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:18.786366 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f633c-030a-47b0-ae1d-16dfe684e725 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:18.804512 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8f633c-030a-47b0-ae1d-16dfe684e725 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:32:18.809980 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-9b8f633c-030a-47b0-ae1d-16dfe684e725 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 15 08:32:18.809980 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:18.809980 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:18.809980 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:18.809980 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:18.809980 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:18.809980 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:18.809980 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:18.809980 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:18.809980 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:18.809980 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:18.809980 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:18.809980 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:32:18.809980 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:32:18.809980 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:32:18.809980 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:32:18.810900 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:32:18.810900 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:32:18.810900 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:32:18.810900 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:32:18.810900 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:18.810900 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:18.810900 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:18.810900 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:18.810900 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:18.810900 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:18.810900 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:18.810900 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:18.810900 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:18.810900 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:18.810900 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:18.810900 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:18.811689 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:18.811689 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:18.811689 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:18.811689 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:18.811689 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:32:18.811689 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:32:18.811689 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:32:18.811689 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:32:18.811689 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:32:18.811689 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:32:18.811689 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 15 08:32:18.811689 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:18.811689 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 182/366] 10.222.0.30 () {56 vars in 1102 bytes} [Sat Oct 15 08:32:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:18.818877 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f73c24a-7c3a-4351-b409-ba0dbc9b4afc None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:18.923929 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:18.978242 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f73c24a-7c3a-4351-b409-ba0dbc9b4afc None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:19.015924 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8f633c-030a-47b0-ae1d-16dfe684e725 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:19.018334 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ce69dc-a75d-4f6c-8331-3b989fd93603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:19.019020 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ce69dc-a75d-4f6c-8331-3b989fd93603 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:19.019305 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ce69dc-a75d-4f6c-8331-3b989fd93603 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:19.019581 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ce69dc-a75d-4f6c-8331-3b989fd93603 None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:19.020254 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ce69dc-a75d-4f6c-8331-3b989fd93603 None admin] RBAC: Adding query filter params (name=service) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:19.020560 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ce69dc-a75d-4f6c-8331-3b989fd93603 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:19.021949 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ce69dc-a75d-4f6c-8331-3b989fd93603 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:19.033207 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 185/367] 10.222.0.30 () {56 vars in 1103 bytes} [Sat Oct 15 08:32:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:19.038322 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ce69dc-a75d-4f6c-8331-3b989fd93603 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:19.101824 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:19.194650 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8f633c-030a-47b0-ae1d-16dfe684e725 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:19.220880 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ce69dc-a75d-4f6c-8331-3b989fd93603 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:19.230782 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84b7e4f-e259-426f-85e2-09e7e56edbf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:19.231419 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b7e4f-e259-426f-85e2-09e7e56edbf9 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:19.231772 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b7e4f-e259-426f-85e2-09e7e56edbf9 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:19.231976 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b7e4f-e259-426f-85e2-09e7e56edbf9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:19.243022 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84b7e4f-e259-426f-85e2-09e7e56edbf9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:32:19.247987 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:19.249736 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-d84b7e4f-e259-426f-85e2-09e7e56edbf9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 15 08:32:19.249736 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:19.249736 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:19.249736 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:19.249736 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:19.249736 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:19.249736 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:19.249736 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:19.249736 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:19.249736 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:19.249736 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:19.249736 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:19.249736 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:32:19.249736 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:32:19.249736 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:32:19.249736 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:32:19.250926 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:32:19.250926 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:32:19.250926 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:32:19.250926 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:32:19.250926 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:19.250926 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:19.250926 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:19.250926 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:19.250926 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:19.250926 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:19.250926 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:19.250926 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:19.250926 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:19.250926 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:19.250926 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:19.250926 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:19.251999 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:19.251999 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:19.251999 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:19.251999 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:19.251999 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:32:19.251999 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:32:19.251999 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:32:19.251999 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:32:19.251999 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:32:19.251999 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:32:19.251999 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 15 08:32:19.251999 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:19.255299 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 183/368] 10.222.0.30 () {56 vars in 1102 bytes} [Sat Oct 15 08:32:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 08:32:19.256965 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d45b6a-32f1-4925-b0ea-40378e37066b None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:19.257341 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d45b6a-32f1-4925-b0ea-40378e37066b None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:19.257790 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d45b6a-32f1-4925-b0ea-40378e37066b None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:19.264105 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 184/369] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:32:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:32:19.264105 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d45b6a-32f1-4925-b0ea-40378e37066b None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:19.270487 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4605a7bf-ef63-483a-8d7a-78c640157ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:19.271829 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4605a7bf-ef63-483a-8d7a-78c640157ba5 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:19.272272 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4605a7bf-ef63-483a-8d7a-78c640157ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:19.272683 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4605a7bf-ef63-483a-8d7a-78c640157ba5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:19.286333 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4605a7bf-ef63-483a-8d7a-78c640157ba5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:19.287870 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4605a7bf-ef63-483a-8d7a-78c640157ba5 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:19.302797 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-4605a7bf-ef63-483a-8d7a-78c640157ba5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:19.302797 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:19.302797 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:19.302797 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:19.302797 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:19.302797 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:19.302797 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:19.302797 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:19.302797 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:19.302797 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:19.302797 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:19.302797 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:19.302797 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 08:32:19.302797 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 08:32:19.302797 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 08:32:19.302797 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 08:32:19.303731 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:19.303731 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:19.303731 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:19.303731 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:19.303731 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:19.303731 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:19.303731 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:19.303731 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:19.303731 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:19.303731 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:19.303731 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:19.303731 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:19.303731 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:19.303731 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:19.303731 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:19.303731 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:19.303731 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 08:32:19.309607 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 08:32:19.309607 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:19.309607 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:19.309607 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 186/370] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:32:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 08:32:19.309607 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d09d19-a9b0-4160-83a4-48dc3c6e86b5 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:19.309607 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d09d19-a9b0-4160-83a4-48dc3c6e86b5 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:19.309607 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d09d19-a9b0-4160-83a4-48dc3c6e86b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:19.326554 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-21d09d19-a9b0-4160-83a4-48dc3c6e86b5 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:32:19.327889 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-21d09d19-a9b0-4160-83a4-48dc3c6e86b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:32:19.381741 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d45b6a-32f1-4925-b0ea-40378e37066b None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:19.383436 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:19.401159 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 187/371] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:32:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 08:32:19.404996 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b857be0a-ef31-4a56-bf1f-a150fd9ecf7a None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:19.405286 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b857be0a-ef31-4a56-bf1f-a150fd9ecf7a None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:19.405565 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b857be0a-ef31-4a56-bf1f-a150fd9ecf7a None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:19.407459 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 188/372] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:32:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:32:19.407984 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b857be0a-ef31-4a56-bf1f-a150fd9ecf7a None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:19.433849 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b4fa88-c8b1-4cc4-8c9c-5b9718a5f031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:19.434767 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b4fa88-c8b1-4cc4-8c9c-5b9718a5f031 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:19.435055 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b4fa88-c8b1-4cc4-8c9c-5b9718a5f031 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:19.435335 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b4fa88-c8b1-4cc4-8c9c-5b9718a5f031 None admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:19.436010 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b4fa88-c8b1-4cc4-8c9c-5b9718a5f031 None admin] RBAC: Adding query filter params (name=service) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:19.436319 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b4fa88-c8b1-4cc4-8c9c-5b9718a5f031 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:19.438219 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b4fa88-c8b1-4cc4-8c9c-5b9718a5f031 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:19.453824 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 185/373] 10.222.0.30 () {56 vars in 1103 bytes} [Sat Oct 15 08:32:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:19.455095 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda356d4-be73-475b-b198-c8d4b19e2dc3 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:19.455396 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda356d4-be73-475b-b198-c8d4b19e2dc3 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:19.455674 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda356d4-be73-475b-b198-c8d4b19e2dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:19.482734 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-fda356d4-be73-475b-b198-c8d4b19e2dc3 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:32:19.484231 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-fda356d4-be73-475b-b198-c8d4b19e2dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:32:19.530331 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b857be0a-ef31-4a56-bf1f-a150fd9ecf7a None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:19.533327 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:19.568164 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34527ac2-357a-4fd5-9f2a-3bad4cf74689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:19.568837 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34527ac2-357a-4fd5-9f2a-3bad4cf74689 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:19.569115 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34527ac2-357a-4fd5-9f2a-3bad4cf74689 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:19.569391 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34527ac2-357a-4fd5-9f2a-3bad4cf74689 None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:19.570104 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34527ac2-357a-4fd5-9f2a-3bad4cf74689 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:19.570411 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34527ac2-357a-4fd5-9f2a-3bad4cf74689 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:19.572350 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34527ac2-357a-4fd5-9f2a-3bad4cf74689 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:19.579427 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 186/374] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:32:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 08:32:19.580366 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda356d4-be73-475b-b198-c8d4b19e2dc3 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:19.592983 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 189/375] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 08:32:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:19.598454 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34527ac2-357a-4fd5-9f2a-3bad4cf74689 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:19.689998 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:19.699488 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda356d4-be73-475b-b198-c8d4b19e2dc3 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:19.746546 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2461e6ce-6ec7-4dee-a126-ea18f7682af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:19.747445 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2461e6ce-6ec7-4dee-a126-ea18f7682af4 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:19.747727 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2461e6ce-6ec7-4dee-a126-ea18f7682af4 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:19.748256 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2461e6ce-6ec7-4dee-a126-ea18f7682af4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:19.751877 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34527ac2-357a-4fd5-9f2a-3bad4cf74689 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:19.761173 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2461e6ce-6ec7-4dee-a126-ea18f7682af4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:19.763555 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2461e6ce-6ec7-4dee-a126-ea18f7682af4 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:19.784910 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-2461e6ce-6ec7-4dee-a126-ea18f7682af4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:19.784910 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:19.784910 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:19.784910 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:19.784910 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:19.784910 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:19.784910 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:19.784910 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:19.784910 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:19.784910 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:19.784910 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:19.784910 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:19.784910 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 08:32:19.784910 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 08:32:19.784910 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 08:32:19.784910 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 08:32:19.786053 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:19.786053 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:19.786053 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:19.786053 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:19.786053 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:19.786053 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:19.786053 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:19.786053 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:19.786053 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:19.786053 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:19.786053 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:19.786053 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:19.786053 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:19.786053 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:19.786053 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:19.786053 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:19.786053 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 08:32:19.787376 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 08:32:19.787376 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:19.787376 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:19.789769 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 187/376] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:32:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:19.790736 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2461e6ce-6ec7-4dee-a126-ea18f7682af4 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:19.795152 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe48ce17-a9f1-4337-8dfd-940cd53bf740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:19.796682 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe48ce17-a9f1-4337-8dfd-940cd53bf740 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:19.796968 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe48ce17-a9f1-4337-8dfd-940cd53bf740 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:19.797248 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe48ce17-a9f1-4337-8dfd-940cd53bf740 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:19.807310 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe48ce17-a9f1-4337-8dfd-940cd53bf740 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:19.809368 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe48ce17-a9f1-4337-8dfd-940cd53bf740 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:19.826832 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-fe48ce17-a9f1-4337-8dfd-940cd53bf740 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 15 08:32:19.826832 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:19.826832 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:19.826832 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:19.826832 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:19.826832 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:19.826832 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:19.826832 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:19.826832 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:19.826832 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:19.826832 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:19.826832 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:19.826832 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 08:32:19.826832 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 08:32:19.826832 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 08:32:19.826832 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 08:32:19.828346 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:19.828346 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:19.828346 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 08:32:19.828346 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 08:32:19.828346 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 08:32:19.828346 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 08:32:19.828346 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:19.828346 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:19.828346 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:19.828346 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:19.828346 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:19.828346 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:19.828346 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:19.828346 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:19.828346 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:19.828346 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:19.828346 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:19.828346 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:19.829930 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:19.829930 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:19.829930 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 08:32:19.829930 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 08:32:19.829930 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 08:32:19.829930 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 08:32:19.829930 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 08:32:19.829930 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 08:32:19.829930 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 15 08:32:19.829930 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:19.829930 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 190/377] 10.222.0.30 () {56 vars in 1093 bytes} [Sat Oct 15 08:32:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 08:32:19.829930 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe48ce17-a9f1-4337-8dfd-940cd53bf740 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:19.862650 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:19.901520 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2461e6ce-6ec7-4dee-a126-ea18f7682af4 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:19.937800 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360b6530-629d-4ba2-b708-0e762c70b515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:19.938538 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b6530-629d-4ba2-b708-0e762c70b515 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:19.938820 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b6530-629d-4ba2-b708-0e762c70b515 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:19.939095 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b6530-629d-4ba2-b708-0e762c70b515 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:19.947209 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe48ce17-a9f1-4337-8dfd-940cd53bf740 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:19.950449 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360b6530-629d-4ba2-b708-0e762c70b515 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:32:19.955762 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-360b6530-629d-4ba2-b708-0e762c70b515 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 15 08:32:19.955762 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:19.955762 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:19.955762 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:19.955762 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:19.955762 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:19.955762 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:19.955762 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:19.955762 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:19.955762 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:19.955762 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:19.955762 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:19.955762 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:32:19.955762 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:32:19.955762 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:32:19.955762 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:32:19.956656 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:32:19.956656 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:32:19.956656 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:32:19.956656 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:32:19.956656 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:19.956656 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:19.956656 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:19.956656 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:19.956656 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:19.956656 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:19.956656 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:19.956656 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:19.956656 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:19.956656 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:19.956656 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:19.956656 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:19.958367 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:19.958367 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:19.958367 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:19.958367 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:19.958367 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:32:19.958367 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:32:19.958367 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:32:19.958367 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:32:19.958367 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:32:19.958367 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:32:19.958367 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 15 08:32:19.958367 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:19.958367 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 188/378] 10.222.0.30 () {56 vars in 1102 bytes} [Sat Oct 15 08:32:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:19.958367 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360b6530-629d-4ba2-b708-0e762c70b515 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:19.967666 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:19.991628 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa56557-1d7c-468b-b16c-345f3ff3c050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:19.992238 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa56557-1d7c-468b-b16c-345f3ff3c050 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:19.992524 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa56557-1d7c-468b-b16c-345f3ff3c050 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:19.992811 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa56557-1d7c-468b-b16c-345f3ff3c050 None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:19.993551 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa56557-1d7c-468b-b16c-345f3ff3c050 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:19.993891 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa56557-1d7c-468b-b16c-345f3ff3c050 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:19.995280 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa56557-1d7c-468b-b16c-345f3ff3c050 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:20.007236 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 191/379] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 08:32:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:20.008063 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa56557-1d7c-468b-b16c-345f3ff3c050 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:20.071150 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360b6530-629d-4ba2-b708-0e762c70b515 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:20.097455 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:20.107239 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbe0d3c-a400-4dfc-b67f-5e74a8aabfa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:20.107844 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe0d3c-a400-4dfc-b67f-5e74a8aabfa0 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:20.108183 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe0d3c-a400-4dfc-b67f-5e74a8aabfa0 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:20.108465 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe0d3c-a400-4dfc-b67f-5e74a8aabfa0 None admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:20.109139 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbe0d3c-a400-4dfc-b67f-5e74a8aabfa0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:20.109446 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbe0d3c-a400-4dfc-b67f-5e74a8aabfa0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:20.110936 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbe0d3c-a400-4dfc-b67f-5e74a8aabfa0 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:20.116328 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa56557-1d7c-468b-b16c-345f3ff3c050 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:20.151155 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 189/380] 10.222.0.30 () {56 vars in 1133 bytes} [Sat Oct 15 08:32:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:20.151836 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbe0d3c-a400-4dfc-b67f-5e74a8aabfa0 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:20.156186 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027854d5-8aae-4486-afc2-6dbc47a7d4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:20.156932 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027854d5-8aae-4486-afc2-6dbc47a7d4b2 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:20.157221 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027854d5-8aae-4486-afc2-6dbc47a7d4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:20.157493 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027854d5-8aae-4486-afc2-6dbc47a7d4b2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:20.169719 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027854d5-8aae-4486-afc2-6dbc47a7d4b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:32:20.174808 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-027854d5-8aae-4486-afc2-6dbc47a7d4b2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 15 08:32:20.174808 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:20.174808 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:20.174808 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:20.174808 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:20.174808 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:20.174808 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:20.174808 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:20.174808 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:20.174808 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:20.174808 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:20.174808 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:20.174808 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:32:20.174808 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:32:20.174808 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:32:20.174808 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:32:20.175703 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:32:20.175703 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:32:20.175703 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:32:20.175703 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:32:20.175703 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:20.175703 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:20.175703 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:20.175703 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:20.175703 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:20.175703 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:20.175703 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:20.175703 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:20.175703 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:20.175703 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:20.175703 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:20.175703 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:20.176560 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:20.176560 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:20.176560 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:20.176560 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:20.176560 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:32:20.176560 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:32:20.176560 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:32:20.176560 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:32:20.176560 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:32:20.176560 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:32:20.176560 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 15 08:32:20.176560 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:20.176560 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 192/381] 10.222.0.30 () {56 vars in 1102 bytes} [Sat Oct 15 08:32:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:20.178038 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027854d5-8aae-4486-afc2-6dbc47a7d4b2 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:20.219802 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:20.263618 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbe0d3c-a400-4dfc-b67f-5e74a8aabfa0 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:20.291245 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027854d5-8aae-4486-afc2-6dbc47a7d4b2 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:20.299642 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0405896-4773-4f28-a183-1bcc9bf8be18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:20.300259 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0405896-4773-4f28-a183-1bcc9bf8be18 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:20.300545 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0405896-4773-4f28-a183-1bcc9bf8be18 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:20.301181 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0405896-4773-4f28-a183-1bcc9bf8be18 None admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:20.301894 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0405896-4773-4f28-a183-1bcc9bf8be18 None admin] RBAC: Adding query filter params (name=service) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:20.302520 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0405896-4773-4f28-a183-1bcc9bf8be18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:20.303917 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0405896-4773-4f28-a183-1bcc9bf8be18 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:20.313171 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 190/382] 10.222.0.30 () {56 vars in 1103 bytes} [Sat Oct 15 08:32:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:20.318349 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0405896-4773-4f28-a183-1bcc9bf8be18 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:20.331975 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a3dde2-88ff-45c6-b3c0-38aa690d6fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:20.332541 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a3dde2-88ff-45c6-b3c0-38aa690d6fe9 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:20.332824 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a3dde2-88ff-45c6-b3c0-38aa690d6fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:20.333103 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a3dde2-88ff-45c6-b3c0-38aa690d6fe9 None admin] PATH_INFO: `/v3/users/glance` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:20.340808 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a3dde2-88ff-45c6-b3c0-38aa690d6fe9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:20.342294 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a3dde2-88ff-45c6-b3c0-38aa690d6fe9 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:20.354181 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-17a3dde2-88ff-45c6-b3c0-38aa690d6fe9 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 15 08:32:20.354181 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:20.354181 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:20.354181 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:20.354181 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:20.354181 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:20.354181 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:20.354181 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:20.354181 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:20.354181 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:20.354181 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:20.354181 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:20.354181 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 08:32:20.354181 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 08:32:20.354181 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 08:32:20.354181 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 08:32:20.355157 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:20.355157 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:20.355157 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 08:32:20.355157 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 08:32:20.355157 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 08:32:20.355157 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 08:32:20.355157 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:20.355157 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:20.355157 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:20.355157 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:20.355157 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:20.355157 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:20.355157 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:20.355157 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:20.355157 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:20.355157 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:20.355157 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:20.355157 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:20.356082 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:20.356082 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:20.356082 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 08:32:20.356082 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 08:32:20.356082 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 08:32:20.356082 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 08:32:20.356082 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 08:32:20.356082 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 08:32:20.356082 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 15 08:32:20.356082 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:20.356082 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 193/383] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 08:32:20 2022] GET /identity/v3/users/glance => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:20.359351 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a3dde2-88ff-45c6-b3c0-38aa690d6fe9 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:20.362717 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:20.420491 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0405896-4773-4f28-a183-1bcc9bf8be18 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:20.444966 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:20.453817 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3afe4e4-dbb5-4ac4-a642-65474337f0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:20.454449 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3afe4e4-dbb5-4ac4-a642-65474337f0bd None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:20.454730 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3afe4e4-dbb5-4ac4-a642-65474337f0bd None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:20.455004 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3afe4e4-dbb5-4ac4-a642-65474337f0bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:20.458723 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a3dde2-88ff-45c6-b3c0-38aa690d6fe9 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:20.465935 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3afe4e4-dbb5-4ac4-a642-65474337f0bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:20.467381 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3afe4e4-dbb5-4ac4-a642-65474337f0bd None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:20.482350 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-a3afe4e4-dbb5-4ac4-a642-65474337f0bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:20.482350 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:20.482350 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:20.482350 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:20.482350 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:20.482350 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:20.482350 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:20.482350 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:20.482350 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:20.482350 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:20.482350 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:20.482350 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:20.482350 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 08:32:20.482350 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 08:32:20.482350 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 08:32:20.482350 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 08:32:20.483331 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:20.483331 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:20.483331 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:20.483331 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:20.483331 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:20.483331 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:20.483331 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:20.483331 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:20.483331 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:20.483331 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:20.483331 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:20.483331 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:20.483331 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:20.483331 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:20.483331 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:20.483331 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:20.483331 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 08:32:20.489061 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 08:32:20.489061 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:20.489061 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:20.489061 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 191/384] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:32:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:20.489061 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3afe4e4-dbb5-4ac4-a642-65474337f0bd None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:20.491556 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f732e99-9b22-45db-85a3-e38091f414d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:20.492159 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f732e99-9b22-45db-85a3-e38091f414d5 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:20.492440 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f732e99-9b22-45db-85a3-e38091f414d5 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:20.492713 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f732e99-9b22-45db-85a3-e38091f414d5 None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:20.493932 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f732e99-9b22-45db-85a3-e38091f414d5 None admin] RBAC: Adding query filter params (name=service) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:20.494237 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f732e99-9b22-45db-85a3-e38091f414d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:20.495595 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f732e99-9b22-45db-85a3-e38091f414d5 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:20.503474 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 194/385] 10.222.0.30 () {56 vars in 1103 bytes} [Sat Oct 15 08:32:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:20.504629 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d18c129-32af-46e6-bd1b-b514c6a2fb1d None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:20.504920 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d18c129-32af-46e6-bd1b-b514c6a2fb1d None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:20.505198 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d18c129-32af-46e6-bd1b-b514c6a2fb1d None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:20.508277 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 195/386] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:32:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:32:20.508277 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d18c129-32af-46e6-bd1b-b514c6a2fb1d None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:20.571984 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:20.590316 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3afe4e4-dbb5-4ac4-a642-65474337f0bd None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:20.607111 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d18c129-32af-46e6-bd1b-b514c6a2fb1d None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:20.622923 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c58145f-b277-4984-8930-33ee65cf72c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:20.623542 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c58145f-b277-4984-8930-33ee65cf72c5 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:20.623822 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c58145f-b277-4984-8930-33ee65cf72c5 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:20.624090 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c58145f-b277-4984-8930-33ee65cf72c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:20.634658 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c58145f-b277-4984-8930-33ee65cf72c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:20.636098 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c58145f-b277-4984-8930-33ee65cf72c5 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:20.640429 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8112d3-7b26-4a53-8855-ce73e58d3742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:20.641022 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8112d3-7b26-4a53-8855-ce73e58d3742 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:20.641306 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8112d3-7b26-4a53-8855-ce73e58d3742 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:20.641582 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8112d3-7b26-4a53-8855-ce73e58d3742 None admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:20.642578 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8112d3-7b26-4a53-8855-ce73e58d3742 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:20.642887 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8112d3-7b26-4a53-8855-ce73e58d3742 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:20.644291 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8112d3-7b26-4a53-8855-ce73e58d3742 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:20.650024 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-8c58145f-b277-4984-8930-33ee65cf72c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:20.650024 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:20.650024 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:20.650024 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:20.650024 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:20.650024 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:20.650024 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:20.650024 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:20.650024 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:20.650024 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:20.650024 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:20.650024 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:20.650024 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 08:32:20.650024 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 08:32:20.650024 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 08:32:20.650024 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 08:32:20.650941 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:20.650941 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:20.650941 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:20.650941 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:20.650941 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:20.650941 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:20.650941 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:20.650941 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:20.650941 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:20.650941 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:20.650941 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:20.650941 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:20.650941 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:20.650941 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:20.650941 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:20.650941 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:20.650941 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 08:32:20.652212 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 08:32:20.652212 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:20.652212 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:20.652212 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 192/387] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:32:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:20.653344 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c58145f-b277-4984-8930-33ee65cf72c5 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:20.680245 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 196/388] 10.222.0.30 () {56 vars in 1137 bytes} [Sat Oct 15 08:32:20 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:20.681711 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8112d3-7b26-4a53-8855-ce73e58d3742 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:20.688526 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:20.757471 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c58145f-b277-4984-8930-33ee65cf72c5 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:20.784704 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8112d3-7b26-4a53-8855-ce73e58d3742 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:20.789856 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cb8625-cd07-448e-bf9f-c188e85ab4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:20.790972 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb8625-cd07-448e-bf9f-c188e85ab4f8 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:20.791257 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb8625-cd07-448e-bf9f-c188e85ab4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:20.791531 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb8625-cd07-448e-bf9f-c188e85ab4f8 None admin] PATH_INFO: `/v3/domains` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:20.792173 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cb8625-cd07-448e-bf9f-c188e85ab4f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:20.792487 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cb8625-cd07-448e-bf9f-c188e85ab4f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:20.793891 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cb8625-cd07-448e-bf9f-c188e85ab4f8 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:20.805378 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 193/389] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 08:32:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:20.806692 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1879e01d-346f-4ba8-98de-6b886a1dd510 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:20.806978 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1879e01d-346f-4ba8-98de-6b886a1dd510 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:20.807256 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1879e01d-346f-4ba8-98de-6b886a1dd510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:20.810442 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:20.819125 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca02a8d-68d0-42a3-a6b1-00a5b06e5f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:20.819760 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca02a8d-68d0-42a3-a6b1-00a5b06e5f08 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:20.820047 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca02a8d-68d0-42a3-a6b1-00a5b06e5f08 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:20.820328 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca02a8d-68d0-42a3-a6b1-00a5b06e5f08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:20.825547 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-1879e01d-346f-4ba8-98de-6b886a1dd510 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:32:20.826772 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-1879e01d-346f-4ba8-98de-6b886a1dd510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:32:20.830724 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca02a8d-68d0-42a3-a6b1-00a5b06e5f08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:20.832206 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca02a8d-68d0-42a3-a6b1-00a5b06e5f08 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:20.846592 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-aca02a8d-68d0-42a3-a6b1-00a5b06e5f08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:20.846592 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:20.846592 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:20.846592 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:20.846592 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:20.846592 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:20.846592 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:20.846592 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:20.846592 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:20.846592 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:20.846592 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:20.846592 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:20.846592 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 08:32:20.846592 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 08:32:20.846592 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 08:32:20.846592 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 08:32:20.847629 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:20.847629 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:20.847629 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:20.847629 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:20.847629 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:20.847629 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:20.847629 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:20.847629 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:20.847629 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:20.847629 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:20.847629 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:20.847629 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:20.847629 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:20.847629 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:20.847629 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:20.847629 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:20.847629 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 08:32:20.848499 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 08:32:20.848499 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:20.848499 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:20.848499 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 197/390] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:32:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:20.853494 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca02a8d-68d0-42a3-a6b1-00a5b06e5f08 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:20.892892 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 194/391] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:32:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 08:32:20.893735 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1879e01d-346f-4ba8-98de-6b886a1dd510 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:20.925235 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:20.953525 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca02a8d-68d0-42a3-a6b1-00a5b06e5f08 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:20.996220 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84343890-41b8-49b7-b94f-db39229ff084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:20.997117 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84343890-41b8-49b7-b94f-db39229ff084 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:20.997541 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84343890-41b8-49b7-b94f-db39229ff084 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:20.997853 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84343890-41b8-49b7-b94f-db39229ff084 None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:20.998487 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84343890-41b8-49b7-b94f-db39229ff084 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:20.998792 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84343890-41b8-49b7-b94f-db39229ff084 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:21.000150 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84343890-41b8-49b7-b94f-db39229ff084 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:21.005381 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1879e01d-346f-4ba8-98de-6b886a1dd510 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:21.015650 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 198/392] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 08:32:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:21.015650 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84343890-41b8-49b7-b94f-db39229ff084 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:21.043544 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f684a4ba-9350-4eda-9d8a-c2a7d54c2a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:21.044172 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684a4ba-9350-4eda-9d8a-c2a7d54c2a18 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:21.044453 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684a4ba-9350-4eda-9d8a-c2a7d54c2a18 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:21.044723 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684a4ba-9350-4eda-9d8a-c2a7d54c2a18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:21.053024 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:21.054923 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f684a4ba-9350-4eda-9d8a-c2a7d54c2a18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:21.056925 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f684a4ba-9350-4eda-9d8a-c2a7d54c2a18 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:21.072953 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-f684a4ba-9350-4eda-9d8a-c2a7d54c2a18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:21.072953 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:21.072953 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:21.072953 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:21.072953 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:21.072953 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:21.072953 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:21.072953 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:21.072953 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:21.072953 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:21.072953 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:21.072953 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:21.072953 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 08:32:21.072953 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 08:32:21.072953 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 08:32:21.072953 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 08:32:21.073880 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:21.073880 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:21.073880 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:21.073880 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:21.073880 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:21.073880 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:21.073880 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:21.073880 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:21.073880 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:21.073880 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:21.073880 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:21.073880 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:21.073880 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:21.073880 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:21.073880 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:21.073880 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:21.073880 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 08:32:21.074926 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 08:32:21.074926 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:21.074926 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:21.078599 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 195/393] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:32:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:21.079312 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f684a4ba-9350-4eda-9d8a-c2a7d54c2a18 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:21.124596 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84343890-41b8-49b7-b94f-db39229ff084 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:21.150648 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:21.162594 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ab729f-d86b-4921-9cb2-3070654186a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:21.163247 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab729f-d86b-4921-9cb2-3070654186a4 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:21.163526 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab729f-d86b-4921-9cb2-3070654186a4 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:21.163803 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab729f-d86b-4921-9cb2-3070654186a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:21.170413 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ab729f-d86b-4921-9cb2-3070654186a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:21.171825 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ab729f-d86b-4921-9cb2-3070654186a4 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:21.179950 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f684a4ba-9350-4eda-9d8a-c2a7d54c2a18 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:21.183729 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-18ab729f-d86b-4921-9cb2-3070654186a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 08:32:21.183729 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:21.183729 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:21.183729 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:21.183729 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:21.183729 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:21.183729 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:21.183729 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:21.183729 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:21.183729 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:21.183729 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:21.183729 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:21.183729 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 08:32:21.183729 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 08:32:21.183729 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 08:32:21.184768 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 08:32:21.184768 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:21.184768 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:21.184768 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:21.184768 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:21.184768 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:21.184768 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:21.184768 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:21.184768 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:21.184768 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:21.184768 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:21.184768 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:21.184768 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:21.184768 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:21.184768 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:21.184768 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:21.184768 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:21.185702 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 08:32:21.185702 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 08:32:21.185702 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 08:32:21.185702 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 08:32:21.185702 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 08:32:21.185702 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 08:32:21.185702 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 08:32:21.185702 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:21.185702 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 199/394] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 08:32:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:21.186194 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ab729f-d86b-4921-9cb2-3070654186a4 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:21.214208 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15fb7fb-a9aa-41b2-8d75-748e9bed9598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:21.214814 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15fb7fb-a9aa-41b2-8d75-748e9bed9598 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:21.215098 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15fb7fb-a9aa-41b2-8d75-748e9bed9598 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:21.215377 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15fb7fb-a9aa-41b2-8d75-748e9bed9598 None admin] PATH_INFO: `/v3/domains` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:21.216022 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15fb7fb-a9aa-41b2-8d75-748e9bed9598 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:21.216330 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15fb7fb-a9aa-41b2-8d75-748e9bed9598 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:21.217725 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15fb7fb-a9aa-41b2-8d75-748e9bed9598 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:21.229142 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 196/395] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 08:32:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:21.229953 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15fb7fb-a9aa-41b2-8d75-748e9bed9598 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:21.281317 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:21.287680 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ab729f-d86b-4921-9cb2-3070654186a4 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:21.320364 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c62d2b3-75d5-461d-aad2-96fceeb1fb14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:21.321156 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c62d2b3-75d5-461d-aad2-96fceeb1fb14 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:21.321477 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c62d2b3-75d5-461d-aad2-96fceeb1fb14 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:21.321774 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c62d2b3-75d5-461d-aad2-96fceeb1fb14 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:21.326982 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15fb7fb-a9aa-41b2-8d75-748e9bed9598 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:21.329669 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c62d2b3-75d5-461d-aad2-96fceeb1fb14 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:21.330490 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c62d2b3-75d5-461d-aad2-96fceeb1fb14 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:21.341318 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-9c62d2b3-75d5-461d-aad2-96fceeb1fb14 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 15 08:32:21.341318 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:21.341318 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:21.341318 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:21.341318 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:21.341318 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:21.341318 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:21.341318 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:21.341318 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:21.341318 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:21.341318 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:21.341318 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:21.341318 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 08:32:21.341318 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 08:32:21.341318 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 08:32:21.341318 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 08:32:21.342442 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:21.342442 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:21.342442 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 08:32:21.342442 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 08:32:21.342442 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 08:32:21.342442 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 08:32:21.342442 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:21.342442 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:21.342442 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:21.342442 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:21.342442 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:21.342442 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:21.342442 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:21.342442 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:21.342442 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:21.342442 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:21.342442 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:21.342442 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:21.343364 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:21.343364 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:21.343364 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 08:32:21.343364 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 08:32:21.343364 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 08:32:21.343364 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 08:32:21.343364 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 08:32:21.343364 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 08:32:21.343364 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 15 08:32:21.343364 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:21.343364 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 200/396] 10.222.0.30 () {56 vars in 1102 bytes} [Sat Oct 15 08:32:21 2022] GET /identity/v3/users/neutron => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:21.343364 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c62d2b3-75d5-461d-aad2-96fceeb1fb14 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:21.360480 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:21.366066 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc69100-d19e-4423-bbef-4ba5cb5ad856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:21.366670 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc69100-d19e-4423-bbef-4ba5cb5ad856 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:21.366959 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc69100-d19e-4423-bbef-4ba5cb5ad856 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:21.367235 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc69100-d19e-4423-bbef-4ba5cb5ad856 None admin] PATH_INFO: `/v3/domains` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:21.367885 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc69100-d19e-4423-bbef-4ba5cb5ad856 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:21.368746 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc69100-d19e-4423-bbef-4ba5cb5ad856 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:21.370175 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc69100-d19e-4423-bbef-4ba5cb5ad856 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:21.381532 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 197/397] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 08:32:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:21.384850 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc69100-d19e-4423-bbef-4ba5cb5ad856 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:21.445689 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c62d2b3-75d5-461d-aad2-96fceeb1fb14 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:21.479040 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc38e2c9-91b2-45cc-b9ab-38459da20429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:21.479827 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38e2c9-91b2-45cc-b9ab-38459da20429 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:21.480418 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38e2c9-91b2-45cc-b9ab-38459da20429 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:21.480708 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38e2c9-91b2-45cc-b9ab-38459da20429 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:21.482719 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc69100-d19e-4423-bbef-4ba5cb5ad856 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:21.490899 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc38e2c9-91b2-45cc-b9ab-38459da20429 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:21.492382 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc38e2c9-91b2-45cc-b9ab-38459da20429 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:21.506075 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-dc38e2c9-91b2-45cc-b9ab-38459da20429 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:21.506075 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:21.506075 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:21.506075 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:21.506075 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:21.506075 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:21.506075 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:21.506075 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:21.506075 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:21.506075 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:21.506075 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:21.506075 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:21.506075 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 08:32:21.506075 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 08:32:21.506075 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 08:32:21.506075 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 08:32:21.506991 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:21.506991 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:21.506991 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:21.506991 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:21.506991 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:21.506991 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:21.506991 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:21.506991 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:21.506991 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:21.506991 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:21.506991 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:21.506991 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:21.506991 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:21.506991 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:21.506991 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:21.506991 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:21.506991 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 08:32:21.507837 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 08:32:21.507837 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:21.507837 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:21.507837 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 201/398] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:32:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:21.508049 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc38e2c9-91b2-45cc-b9ab-38459da20429 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:21.519901 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:21.521222 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fcdba1-9dbf-41e7-ace1-08906027e220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:21.521909 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fcdba1-9dbf-41e7-ace1-08906027e220 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:21.522288 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fcdba1-9dbf-41e7-ace1-08906027e220 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:21.522568 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fcdba1-9dbf-41e7-ace1-08906027e220 None admin] PATH_INFO: `/v3/projects` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:21.523290 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fcdba1-9dbf-41e7-ace1-08906027e220 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:21.523577 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fcdba1-9dbf-41e7-ace1-08906027e220 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:21.525110 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fcdba1-9dbf-41e7-ace1-08906027e220 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:21.548552 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 198/399] 10.222.0.30 () {56 vars in 1148 bytes} [Sat Oct 15 08:32:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:21.548552 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fcdba1-9dbf-41e7-ace1-08906027e220 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:21.601471 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:21.608836 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc38e2c9-91b2-45cc-b9ab-38459da20429 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:21.643442 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280d91c5-2420-48fb-9030-d9f64888cd02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:21.643995 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d91c5-2420-48fb-9030-d9f64888cd02 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:21.644275 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d91c5-2420-48fb-9030-d9f64888cd02 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:21.644543 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d91c5-2420-48fb-9030-d9f64888cd02 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:21.647562 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fcdba1-9dbf-41e7-ace1-08906027e220 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:21.651496 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280d91c5-2420-48fb-9030-d9f64888cd02 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:21.652930 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280d91c5-2420-48fb-9030-d9f64888cd02 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:21.664471 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-280d91c5-2420-48fb-9030-d9f64888cd02 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 15 08:32:21.664471 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:21.664471 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:21.664471 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:21.664471 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:21.664471 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:21.664471 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:21.664471 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:21.664471 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:21.664471 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:21.664471 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:21.664471 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:21.664471 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 08:32:21.664471 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 08:32:21.664471 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 08:32:21.664471 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 08:32:21.665942 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:21.665942 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:21.665942 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 08:32:21.665942 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 08:32:21.665942 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 08:32:21.665942 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 08:32:21.665942 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:21.665942 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:21.665942 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:21.665942 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:21.665942 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:21.665942 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:21.665942 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:21.665942 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:21.665942 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:21.665942 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:21.665942 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:21.665942 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:21.666831 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:21.666831 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:21.666831 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 08:32:21.666831 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 08:32:21.666831 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 08:32:21.666831 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 08:32:21.666831 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 08:32:21.666831 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 08:32:21.666831 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 15 08:32:21.666831 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:21.666831 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 202/400] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 08:32:21 2022] GET /identity/v3/users/cinder => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:21.667987 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280d91c5-2420-48fb-9030-d9f64888cd02 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:21.684530 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc564925-406c-49d2-8aee-ac879aee1117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:21.685134 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc564925-406c-49d2-8aee-ac879aee1117 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:21.685412 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc564925-406c-49d2-8aee-ac879aee1117 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:21.685700 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc564925-406c-49d2-8aee-ac879aee1117 None admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:21.686438 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc564925-406c-49d2-8aee-ac879aee1117 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:21.686746 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc564925-406c-49d2-8aee-ac879aee1117 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:21.688188 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc564925-406c-49d2-8aee-ac879aee1117 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:21.705365 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 199/401] 10.222.0.30 () {56 vars in 1139 bytes} [Sat Oct 15 08:32:21 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:21.706044 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc564925-406c-49d2-8aee-ac879aee1117 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:21.734474 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:21.768024 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280d91c5-2420-48fb-9030-d9f64888cd02 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:21.801129 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ce060-09be-4d7d-8596-e38e3f9ff4da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:21.801769 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ce060-09be-4d7d-8596-e38e3f9ff4da None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:21.801927 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc564925-406c-49d2-8aee-ac879aee1117 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:21.802055 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ce060-09be-4d7d-8596-e38e3f9ff4da None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:21.802312 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ce060-09be-4d7d-8596-e38e3f9ff4da None admin] PATH_INFO: `/v3/projects/service` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:21.808835 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18ce060-09be-4d7d-8596-e38e3f9ff4da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:21.810306 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18ce060-09be-4d7d-8596-e38e3f9ff4da None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:21.812682 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:21.821512 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-e18ce060-09be-4d7d-8596-e38e3f9ff4da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 08:32:21.821512 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:21.821512 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:21.821512 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:21.821512 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:21.821512 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:21.821512 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:21.821512 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:21.821512 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:21.821512 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:21.821512 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:21.821512 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:21.821512 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 08:32:21.821512 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 08:32:21.821512 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 08:32:21.822510 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 08:32:21.822510 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:21.822510 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:21.822510 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:21.822510 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:21.822510 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:21.822510 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:21.822510 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:21.822510 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:21.822510 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:21.822510 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:21.822510 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:21.822510 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:21.822510 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:21.822510 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:21.822510 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:21.822510 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:21.823363 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 08:32:21.823363 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 08:32:21.823363 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 08:32:21.823363 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 08:32:21.823363 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 08:32:21.823363 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 08:32:21.823363 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 08:32:21.823363 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:21.823363 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 203/402] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 08:32:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:21.823363 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ce060-09be-4d7d-8596-e38e3f9ff4da None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:21.839052 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2884dc70-5541-438b-bb26-833b4a3f4167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:21.839659 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2884dc70-5541-438b-bb26-833b4a3f4167 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:21.839944 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2884dc70-5541-438b-bb26-833b4a3f4167 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:21.840220 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2884dc70-5541-438b-bb26-833b4a3f4167 None admin] PATH_INFO: `/v3/domains` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:21.840867 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2884dc70-5541-438b-bb26-833b4a3f4167 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:21.841174 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2884dc70-5541-438b-bb26-833b4a3f4167 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:21.842574 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2884dc70-5541-438b-bb26-833b4a3f4167 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:21.854181 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 200/403] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 08:32:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:21.855245 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2884dc70-5541-438b-bb26-833b4a3f4167 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:21.924904 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ce060-09be-4d7d-8596-e38e3f9ff4da None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:21.950465 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:21.953701 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2884dc70-5541-438b-bb26-833b4a3f4167 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:21.958768 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09385af9-4c64-492a-a4e0-6a2675af943e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:21.959423 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09385af9-4c64-492a-a4e0-6a2675af943e None admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:21.959708 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09385af9-4c64-492a-a4e0-6a2675af943e None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:21.959991 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09385af9-4c64-492a-a4e0-6a2675af943e None admin] PATH_INFO: `/v3/projects/5dfceb59b3794439a4586b3cbe408f0e/users/af0a8af5cadc4cc1ab73e8eb2a20bb7e/roles/b1b0b29beb794324826a4c4003ae293b` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:21.974021 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09385af9-4c64-492a-a4e0-6a2675af943e None admin] RBAC: Authorizing `identity:create_grant(project_id=5dfceb59b3794439a4586b3cbe408f0e, user_id=af0a8af5cadc4cc1ab73e8eb2a20bb7e, role_id=b1b0b29beb794324826a4c4003ae293b)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:21.975504 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09385af9-4c64-492a-a4e0-6a2675af943e None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:21.986495 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67003b5-c791-4b18-a94f-4d4af26e1c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:21.987096 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67003b5-c791-4b18-a94f-4d4af26e1c4c None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:21.987378 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67003b5-c791-4b18-a94f-4d4af26e1c4c None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:21.987671 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67003b5-c791-4b18-a94f-4d4af26e1c4c None admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:21.987919 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 204/404] 10.222.0.30 () {58 vars in 1439 bytes} [Sat Oct 15 08:32:21 2022] PUT /identity/v3/projects/5dfceb59b3794439a4586b3cbe408f0e/users/af0a8af5cadc4cc1ab73e8eb2a20bb7e/roles/b1b0b29beb794324826a4c4003ae293b => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:32:21.988372 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67003b5-c791-4b18-a94f-4d4af26e1c4c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:21.988680 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67003b5-c791-4b18-a94f-4d4af26e1c4c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:21.988771 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09385af9-4c64-492a-a4e0-6a2675af943e None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:21.990138 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67003b5-c791-4b18-a94f-4d4af26e1c4c None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:22.014463 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 201/405] 10.222.0.30 () {56 vars in 1137 bytes} [Sat Oct 15 08:32:21 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:22.014463 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67003b5-c791-4b18-a94f-4d4af26e1c4c None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:22.049599 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:22.116969 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09385af9-4c64-492a-a4e0-6a2675af943e None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:22.132081 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67003b5-c791-4b18-a94f-4d4af26e1c4c None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:22.158555 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bf9cbc-69d9-468f-ad34-49e95da8d4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:22.159814 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bf9cbc-69d9-468f-ad34-49e95da8d4f7 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:22.159814 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bf9cbc-69d9-468f-ad34-49e95da8d4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:22.159814 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bf9cbc-69d9-468f-ad34-49e95da8d4f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:22.170663 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bf9cbc-69d9-468f-ad34-49e95da8d4f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:22.172128 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bf9cbc-69d9-468f-ad34-49e95da8d4f7 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:22.176122 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc183f5-9d06-4fc9-b461-a8763e04b2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:22.176675 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc183f5-9d06-4fc9-b461-a8763e04b2f5 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:22.176959 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc183f5-9d06-4fc9-b461-a8763e04b2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:22.177236 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc183f5-9d06-4fc9-b461-a8763e04b2f5 None admin] PATH_INFO: `/v3/projects` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:22.177984 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc183f5-9d06-4fc9-b461-a8763e04b2f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:22.178293 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc183f5-9d06-4fc9-b461-a8763e04b2f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:22.179742 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc183f5-9d06-4fc9-b461-a8763e04b2f5 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:22.187141 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-a4bf9cbc-69d9-468f-ad34-49e95da8d4f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:22.187141 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:22.187141 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:22.187141 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:22.187141 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:22.187141 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:22.187141 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:22.187141 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:22.187141 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:22.187141 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:22.187141 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:22.187141 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:22.187141 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 08:32:22.187141 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 08:32:22.187141 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 08:32:22.187141 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 08:32:22.188282 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:22.188282 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:22.188282 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:22.188282 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:22.188282 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:22.188282 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:22.188282 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:22.188282 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:22.188282 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:22.188282 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:22.188282 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:22.188282 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:22.188282 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:22.188282 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:22.188282 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:22.188282 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:22.188282 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 08:32:22.189196 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 08:32:22.189196 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:22.189196 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:22.191308 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 205/406] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:32:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:22.195848 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bf9cbc-69d9-468f-ad34-49e95da8d4f7 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:22.195971 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 202/407] 10.222.0.30 () {56 vars in 1148 bytes} [Sat Oct 15 08:32:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:22.195971 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc183f5-9d06-4fc9-b461-a8763e04b2f5 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:22.230739 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:22.291088 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:22.305284 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc183f5-9d06-4fc9-b461-a8763e04b2f5 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:22.310886 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bf9cbc-69d9-468f-ad34-49e95da8d4f7 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:22.342522 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91466ed6-1ef2-49de-85d3-7459275ccf48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:22.343160 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91466ed6-1ef2-49de-85d3-7459275ccf48 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:22.343441 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91466ed6-1ef2-49de-85d3-7459275ccf48 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:22.343715 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91466ed6-1ef2-49de-85d3-7459275ccf48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:22.349229 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb65534-ceac-45ec-b5e8-24fc371c32fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:22.350133 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb65534-ceac-45ec-b5e8-24fc371c32fc None admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:22.350671 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb65534-ceac-45ec-b5e8-24fc371c32fc None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:22.351129 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb65534-ceac-45ec-b5e8-24fc371c32fc None admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:22.351952 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb65534-ceac-45ec-b5e8-24fc371c32fc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:22.354947 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91466ed6-1ef2-49de-85d3-7459275ccf48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:22.355324 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb65534-ceac-45ec-b5e8-24fc371c32fc None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:22.356500 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91466ed6-1ef2-49de-85d3-7459275ccf48 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:22.371317 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-91466ed6-1ef2-49de-85d3-7459275ccf48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:22.371317 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:22.371317 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:22.371317 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:22.371317 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:22.371317 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:22.371317 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:22.371317 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:22.371317 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:22.371317 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:22.371317 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:22.371317 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:22.371317 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 08:32:22.371317 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 08:32:22.371317 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 08:32:22.371317 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 08:32:22.372713 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:22.372713 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:22.372713 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:22.372713 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:22.372713 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:22.372713 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:22.372713 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:22.372713 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:22.372713 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:22.372713 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:22.372713 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:22.372713 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:22.372713 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:22.372713 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:22.372713 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:22.372713 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:22.372713 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 08:32:22.373612 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 08:32:22.373612 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:22.373612 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:22.373612 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 203/408] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:32:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:22.373612 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91466ed6-1ef2-49de-85d3-7459275ccf48 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:22.381445 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 206/409] 10.222.0.30 () {60 vars in 1132 bytes} [Sat Oct 15 08:32:22 2022] POST /identity/v3/users => generated 247 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:32:22.382520 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb65534-ceac-45ec-b5e8-24fc371c32fc None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:22.514031 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:22.525616 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91466ed6-1ef2-49de-85d3-7459275ccf48 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:22.541555 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb65534-ceac-45ec-b5e8-24fc371c32fc None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:22.575875 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbff7921-3f9d-48a4-9c14-ccc3b25c3e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:22.576490 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbff7921-3f9d-48a4-9c14-ccc3b25c3e9d None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:22.576780 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbff7921-3f9d-48a4-9c14-ccc3b25c3e9d None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:22.577057 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbff7921-3f9d-48a4-9c14-ccc3b25c3e9d None admin] PATH_INFO: `/v3/domains` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:22.577724 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbff7921-3f9d-48a4-9c14-ccc3b25c3e9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:22.578039 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbff7921-3f9d-48a4-9c14-ccc3b25c3e9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:22.579426 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbff7921-3f9d-48a4-9c14-ccc3b25c3e9d None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:22.591097 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:22.591242 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34ffde5-3364-4040-890f-2e9c44ba0b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:22.591587 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34ffde5-3364-4040-890f-2e9c44ba0b8a None admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:22.592256 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34ffde5-3364-4040-890f-2e9c44ba0b8a None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:22.592525 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34ffde5-3364-4040-890f-2e9c44ba0b8a None admin] PATH_INFO: `/v3/projects/5dfceb59b3794439a4586b3cbe408f0e/users/71692d05fd324679b1598feb52b3fe6d/roles/b1b0b29beb794324826a4c4003ae293b` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:22.594324 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 204/410] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 08:32:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:22.595165 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbff7921-3f9d-48a4-9c14-ccc3b25c3e9d None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:22.610972 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34ffde5-3364-4040-890f-2e9c44ba0b8a None admin] RBAC: Authorizing `identity:create_grant(project_id=5dfceb59b3794439a4586b3cbe408f0e, user_id=71692d05fd324679b1598feb52b3fe6d, role_id=b1b0b29beb794324826a4c4003ae293b)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:22.612817 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34ffde5-3364-4040-890f-2e9c44ba0b8a None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:22.627350 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 207/411] 10.222.0.30 () {58 vars in 1439 bytes} [Sat Oct 15 08:32:22 2022] PUT /identity/v3/projects/5dfceb59b3794439a4586b3cbe408f0e/users/71692d05fd324679b1598feb52b3fe6d/roles/b1b0b29beb794324826a4c4003ae293b => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:32:22.628268 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34ffde5-3364-4040-890f-2e9c44ba0b8a None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:22.762129 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbff7921-3f9d-48a4-9c14-ccc3b25c3e9d None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:22.811745 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34ffde5-3364-4040-890f-2e9c44ba0b8a None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:22.838422 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7bec36-7f2d-4209-8891-e8f2b9a9d7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:22.839110 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7bec36-7f2d-4209-8891-e8f2b9a9d7ab None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:22.839399 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7bec36-7f2d-4209-8891-e8f2b9a9d7ab None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:22.839780 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7bec36-7f2d-4209-8891-e8f2b9a9d7ab None admin] PATH_INFO: `/v3/domains` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:22.840473 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7bec36-7f2d-4209-8891-e8f2b9a9d7ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:22.840851 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7bec36-7f2d-4209-8891-e8f2b9a9d7ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:22.842270 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7bec36-7f2d-4209-8891-e8f2b9a9d7ab None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:22.851874 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031d712f-1864-4ee1-87a4-7a19b559d647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:22.853051 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031d712f-1864-4ee1-87a4-7a19b559d647 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:22.853338 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031d712f-1864-4ee1-87a4-7a19b559d647 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:22.853607 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031d712f-1864-4ee1-87a4-7a19b559d647 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:22.856435 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:22.858045 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 205/412] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 08:32:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:22.865700 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031d712f-1864-4ee1-87a4-7a19b559d647 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:22.865700 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031d712f-1864-4ee1-87a4-7a19b559d647 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:22.868768 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7bec36-7f2d-4209-8891-e8f2b9a9d7ab None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:22.876814 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-031d712f-1864-4ee1-87a4-7a19b559d647 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 08:32:22.876814 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:22.876814 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:22.876814 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:22.876814 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:22.876814 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:22.876814 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:22.876814 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:22.876814 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:22.876814 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:22.876814 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:22.876814 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:22.876814 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 08:32:22.876814 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 08:32:22.876814 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 08:32:22.877664 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 08:32:22.877664 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:22.877664 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:22.877664 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:22.877664 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:22.877664 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:22.877664 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:22.877664 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:22.877664 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:22.877664 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:22.877664 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:22.877664 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:22.877664 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:22.877664 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:22.877664 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:22.877664 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:22.877664 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:22.878584 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 08:32:22.878584 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 08:32:22.878584 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 08:32:22.878584 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 08:32:22.878584 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 08:32:22.878584 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 08:32:22.878584 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 08:32:22.878584 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:22.878584 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 208/413] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 08:32:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:22.886661 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031d712f-1864-4ee1-87a4-7a19b559d647 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:22.959475 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:23.032946 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7bec36-7f2d-4209-8891-e8f2b9a9d7ab None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:23.052355 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031d712f-1864-4ee1-87a4-7a19b559d647 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:23.095017 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e947fe25-ea5f-434e-b8bc-37f8c4aa1361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:23.095993 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e947fe25-ea5f-434e-b8bc-37f8c4aa1361 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:23.096589 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e947fe25-ea5f-434e-b8bc-37f8c4aa1361 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:23.097204 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e947fe25-ea5f-434e-b8bc-37f8c4aa1361 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:23.102187 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dbd79a-7aaa-4bab-985c-91d6ace8ae0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:23.103325 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dbd79a-7aaa-4bab-985c-91d6ace8ae0a None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:23.103609 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dbd79a-7aaa-4bab-985c-91d6ace8ae0a None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:23.103884 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dbd79a-7aaa-4bab-985c-91d6ace8ae0a None admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:23.104738 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dbd79a-7aaa-4bab-985c-91d6ace8ae0a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:23.105053 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dbd79a-7aaa-4bab-985c-91d6ace8ae0a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:23.106827 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dbd79a-7aaa-4bab-985c-91d6ace8ae0a None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:23.113072 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e947fe25-ea5f-434e-b8bc-37f8c4aa1361 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:23.114817 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e947fe25-ea5f-434e-b8bc-37f8c4aa1361 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:23.125904 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 209/414] 10.222.0.30 () {56 vars in 1148 bytes} [Sat Oct 15 08:32:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 08:32:23.130802 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-e947fe25-ea5f-434e-b8bc-37f8c4aa1361 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 08:32:23.130802 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:23.130802 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:23.130802 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:23.130802 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:23.130802 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:23.130802 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:23.130802 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:23.130802 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:23.130802 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:23.130802 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:23.130802 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:23.130802 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 08:32:23.130802 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 08:32:23.130802 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 08:32:23.131726 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 08:32:23.131726 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:23.131726 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:23.131726 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:23.131726 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:23.131726 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:23.131726 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:23.131726 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:23.131726 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:23.131726 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:23.131726 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:23.131726 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:23.131726 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:23.131726 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:23.131726 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:23.131726 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:23.131726 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:23.132603 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 08:32:23.132603 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 08:32:23.132603 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 08:32:23.132603 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 08:32:23.132603 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 08:32:23.132603 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 08:32:23.132603 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 08:32:23.132603 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:23.134660 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 206/415] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 08:32:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:23.135677 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dbd79a-7aaa-4bab-985c-91d6ace8ae0a None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:23.144908 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e947fe25-ea5f-434e-b8bc-37f8c4aa1361 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:23.213301 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:23.268844 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dbd79a-7aaa-4bab-985c-91d6ace8ae0a None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:23.321876 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17d69c4-f740-4db8-acba-460a103de3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:23.322563 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17d69c4-f740-4db8-acba-460a103de3b7 None admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:23.322846 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17d69c4-f740-4db8-acba-460a103de3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:23.323122 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17d69c4-f740-4db8-acba-460a103de3b7 None admin] PATH_INFO: `/v3/projects/5dfceb59b3794439a4586b3cbe408f0e/users/8b11e349e4cb4558a4c75d843effc0cd/roles/b1b0b29beb794324826a4c4003ae293b` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:23.351194 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17d69c4-f740-4db8-acba-460a103de3b7 None admin] RBAC: Authorizing `identity:create_grant(project_id=5dfceb59b3794439a4586b3cbe408f0e, user_id=8b11e349e4cb4558a4c75d843effc0cd, role_id=b1b0b29beb794324826a4c4003ae293b)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:23.352601 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17d69c4-f740-4db8-acba-460a103de3b7 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:23.369870 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e947fe25-ea5f-434e-b8bc-37f8c4aa1361 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:23.377845 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 210/416] 10.222.0.30 () {58 vars in 1439 bytes} [Sat Oct 15 08:32:23 2022] PUT /identity/v3/projects/5dfceb59b3794439a4586b3cbe408f0e/users/8b11e349e4cb4558a4c75d843effc0cd/roles/b1b0b29beb794324826a4c4003ae293b => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:32:23.413287 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:23.437041 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4e79ca-59ca-40fb-ace5-0f9cfd6f56b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:23.437698 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4e79ca-59ca-40fb-ace5-0f9cfd6f56b7 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:23.437980 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4e79ca-59ca-40fb-ace5-0f9cfd6f56b7 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:23.438262 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4e79ca-59ca-40fb-ace5-0f9cfd6f56b7 None admin] PATH_INFO: `/v3/projects` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:23.438978 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4e79ca-59ca-40fb-ace5-0f9cfd6f56b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:23.439288 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4e79ca-59ca-40fb-ace5-0f9cfd6f56b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:23.440763 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4e79ca-59ca-40fb-ace5-0f9cfd6f56b7 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:23.459573 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 207/417] 10.222.0.30 () {56 vars in 1148 bytes} [Sat Oct 15 08:32:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 08:32:23.465620 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17d69c4-f740-4db8-acba-460a103de3b7 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:23.593662 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17d69c4-f740-4db8-acba-460a103de3b7 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:23.602961 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:23.639530 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226d02aa-83df-4086-8d77-5a004ad98ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:23.640381 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d02aa-83df-4086-8d77-5a004ad98ce3 None admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:23.640628 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d02aa-83df-4086-8d77-5a004ad98ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:23.640918 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d02aa-83df-4086-8d77-5a004ad98ce3 None admin] PATH_INFO: `/v3/projects/5dfceb59b3794439a4586b3cbe408f0e/users/a649c5fedad847fb9e784e779da3a24e/roles/b1b0b29beb794324826a4c4003ae293b` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:23.661868 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226d02aa-83df-4086-8d77-5a004ad98ce3 None admin] RBAC: Authorizing `identity:create_grant(project_id=5dfceb59b3794439a4586b3cbe408f0e, user_id=a649c5fedad847fb9e784e779da3a24e, role_id=b1b0b29beb794324826a4c4003ae293b)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:23.663413 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226d02aa-83df-4086-8d77-5a004ad98ce3 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:23.678531 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 211/418] 10.222.0.30 () {58 vars in 1439 bytes} [Sat Oct 15 08:32:23 2022] PUT /identity/v3/projects/5dfceb59b3794439a4586b3cbe408f0e/users/a649c5fedad847fb9e784e779da3a24e/roles/b1b0b29beb794324826a4c4003ae293b => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:32:24.290361 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6984789f-3b49-421e-b747-b60b2a2f9c2c None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:24.290737 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6984789f-3b49-421e-b747-b60b2a2f9c2c None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:24.290921 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6984789f-3b49-421e-b747-b60b2a2f9c2c None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:24.292570 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 208/419] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:32:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:32:24.307705 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b3783f-da24-4457-b6e7-a2add7f97437 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:24.308001 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b3783f-da24-4457-b6e7-a2add7f97437 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:24.308348 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b3783f-da24-4457-b6e7-a2add7f97437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:24.338689 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-17b3783f-da24-4457-b6e7-a2add7f97437 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:32:24.339905 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-17b3783f-da24-4457-b6e7-a2add7f97437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:32:24.425392 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 212/420] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:32:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 08:32:24.751658 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6984789f-3b49-421e-b747-b60b2a2f9c2c None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:24.870128 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6984789f-3b49-421e-b747-b60b2a2f9c2c None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:24.915661 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131a2be5-ad2d-48ab-857b-1efe846b5f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:24.916651 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a2be5-ad2d-48ab-857b-1efe846b5f6d None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:24.917137 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a2be5-ad2d-48ab-857b-1efe846b5f6d None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:24.917635 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a2be5-ad2d-48ab-857b-1efe846b5f6d None admin] PATH_INFO: `/v3/roles/service` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:24.928488 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131a2be5-ad2d-48ab-857b-1efe846b5f6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:32:24.934147 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-131a2be5-ad2d-48ab-857b-1efe846b5f6d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 15 08:32:24.934147 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:24.934147 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:24.934147 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:24.934147 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:24.934147 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:24.934147 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:24.934147 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:24.934147 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:24.934147 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:24.934147 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:24.934147 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:24.934147 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:32:24.934147 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:32:24.934147 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:32:24.934147 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:32:24.935034 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:32:24.935034 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:32:24.935034 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:32:24.935034 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:32:24.935034 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:24.935034 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:24.935034 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:24.935034 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:24.935034 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:24.935034 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:24.935034 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:24.935034 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:24.935034 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:24.935034 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:24.935034 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:24.935034 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:24.935864 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:24.935864 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:24.935864 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:24.935864 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:24.935864 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:32:24.935864 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:32:24.935864 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:32:24.935864 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:32:24.935864 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:32:24.935864 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:32:24.935864 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 15 08:32:24.935864 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:24.938459 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 209/421] 10.222.0.30 () {56 vars in 1102 bytes} [Sat Oct 15 08:32:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:24.944169 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b3783f-da24-4457-b6e7-a2add7f97437 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:25.074222 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:25.089617 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b3783f-da24-4457-b6e7-a2add7f97437 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:25.125400 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b697ce-65f0-48b4-8401-e6cb046c1880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:25.126041 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b697ce-65f0-48b4-8401-e6cb046c1880 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:25.126322 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b697ce-65f0-48b4-8401-e6cb046c1880 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:25.126600 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b697ce-65f0-48b4-8401-e6cb046c1880 None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:25.127268 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b697ce-65f0-48b4-8401-e6cb046c1880 None admin] RBAC: Adding query filter params (name=service) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:25.127573 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b697ce-65f0-48b4-8401-e6cb046c1880 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:25.128947 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b697ce-65f0-48b4-8401-e6cb046c1880 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:25.138260 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 213/422] 10.222.0.30 () {56 vars in 1103 bytes} [Sat Oct 15 08:32:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:25.144156 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131a2be5-ad2d-48ab-857b-1efe846b5f6d None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:25.194112 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:25.234199 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e2276-efc8-402f-8393-020cd2916317 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:25.234422 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e2276-efc8-402f-8393-020cd2916317 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:25.234629 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e2276-efc8-402f-8393-020cd2916317 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:25.236305 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 214/423] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:32:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:32:25.241257 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32017557-d036-4f9f-92e1-abe07d06861f None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:25.241564 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32017557-d036-4f9f-92e1-abe07d06861f None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:25.241854 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32017557-d036-4f9f-92e1-abe07d06861f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:25.260765 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131a2be5-ad2d-48ab-857b-1efe846b5f6d None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:25.282922 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-32017557-d036-4f9f-92e1-abe07d06861f None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:32:25.284160 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-32017557-d036-4f9f-92e1-abe07d06861f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:32:25.309112 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f7db34-1818-485c-9060-20ac5bae7fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:25.310122 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f7db34-1818-485c-9060-20ac5bae7fba None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:25.310630 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f7db34-1818-485c-9060-20ac5bae7fba None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:25.311103 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f7db34-1818-485c-9060-20ac5bae7fba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:25.331532 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f7db34-1818-485c-9060-20ac5bae7fba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:25.332960 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f7db34-1818-485c-9060-20ac5bae7fba None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:25.359149 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-d5f7db34-1818-485c-9060-20ac5bae7fba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:25.359149 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:25.359149 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:25.359149 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:25.359149 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:25.359149 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:25.359149 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:25.359149 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:25.359149 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:25.359149 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:25.359149 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:25.359149 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:25.359149 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 08:32:25.359149 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 08:32:25.359149 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 08:32:25.359149 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 08:32:25.360121 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:25.360121 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:25.360121 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:25.360121 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:25.360121 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:25.360121 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:25.360121 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:25.360121 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:25.360121 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:25.360121 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:25.360121 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:25.360121 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:25.360121 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:25.360121 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:25.360121 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:25.360121 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:25.360121 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 08:32:25.360968 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 08:32:25.360968 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:25.360968 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:25.364997 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 210/424] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:32:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:25.366785 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f7db34-1818-485c-9060-20ac5bae7fba None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:25.367642 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 215/425] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:32:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 08:32:25.387496 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e53fa4-1ad4-480f-b096-33e8220a3e89 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:25.387800 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e53fa4-1ad4-480f-b096-33e8220a3e89 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:25.388068 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e53fa4-1ad4-480f-b096-33e8220a3e89 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:25.393722 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 216/426] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:32:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 08:32:25.397075 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bf626a-efc9-4e56-bc2f-bdfa945e40f7 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:25.397075 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bf626a-efc9-4e56-bc2f-bdfa945e40f7 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:25.397075 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bf626a-efc9-4e56-bc2f-bdfa945e40f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:25.416710 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-c7bf626a-efc9-4e56-bc2f-bdfa945e40f7 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:32:25.418268 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-c7bf626a-efc9-4e56-bc2f-bdfa945e40f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:32:25.508295 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 217/427] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:32:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 08:32:25.542031 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:25.548115 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f7db34-1818-485c-9060-20ac5bae7fba None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:25.583952 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3480ebb-179a-402a-9a42-5712561d39e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:25.584486 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3480ebb-179a-402a-9a42-5712561d39e4 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:25.584772 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3480ebb-179a-402a-9a42-5712561d39e4 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:25.585050 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3480ebb-179a-402a-9a42-5712561d39e4 None admin] PATH_INFO: `/v3/domains` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:25.585710 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3480ebb-179a-402a-9a42-5712561d39e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:25.586023 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3480ebb-179a-402a-9a42-5712561d39e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:25.587992 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3480ebb-179a-402a-9a42-5712561d39e4 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:25.601107 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 211/428] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 08:32:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:25.606384 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bf626a-efc9-4e56-bc2f-bdfa945e40f7 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:25.616999 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:25.676565 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3480ebb-179a-402a-9a42-5712561d39e4 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:25.769252 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bf626a-efc9-4e56-bc2f-bdfa945e40f7 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:25.829381 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38e15e3-6555-4d8a-9910-a69efe382b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:25.830053 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e15e3-6555-4d8a-9910-a69efe382b1e None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:25.830337 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e15e3-6555-4d8a-9910-a69efe382b1e None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:25.830612 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e15e3-6555-4d8a-9910-a69efe382b1e None admin] PATH_INFO: `/v3/users/nova` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:25.840776 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38e15e3-6555-4d8a-9910-a69efe382b1e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:25.842163 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38e15e3-6555-4d8a-9910-a69efe382b1e None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:25.846590 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3480ebb-179a-402a-9a42-5712561d39e4 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:25.854286 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-c38e15e3-6555-4d8a-9910-a69efe382b1e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 15 08:32:25.854286 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:25.854286 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:25.854286 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:25.854286 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:25.854286 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:25.854286 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:25.854286 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:25.854286 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:25.854286 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:25.854286 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:25.854286 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:25.854286 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 08:32:25.854286 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 08:32:25.854286 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 08:32:25.854286 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 08:32:25.855172 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:25.855172 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:25.855172 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 08:32:25.855172 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 08:32:25.855172 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 08:32:25.855172 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 08:32:25.855172 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:25.855172 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:25.855172 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:25.855172 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:25.855172 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:25.855172 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:25.855172 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:25.855172 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:25.855172 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:25.855172 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:25.855172 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:25.855172 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:25.856065 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:25.856065 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:25.856065 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 08:32:25.856065 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 08:32:25.856065 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 08:32:25.856065 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 08:32:25.856065 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 08:32:25.856065 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 08:32:25.856065 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 15 08:32:25.856065 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:25.856065 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 218/429] 10.222.0.30 () {56 vars in 1093 bytes} [Sat Oct 15 08:32:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 08:32:25.856624 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38e15e3-6555-4d8a-9910-a69efe382b1e None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:25.889183 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de459b6a-4457-4cb3-a6db-2ca1890d594a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:25.890230 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de459b6a-4457-4cb3-a6db-2ca1890d594a None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:25.890711 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de459b6a-4457-4cb3-a6db-2ca1890d594a None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:25.891241 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de459b6a-4457-4cb3-a6db-2ca1890d594a None admin] PATH_INFO: `/v3/roles/service` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:25.903017 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de459b6a-4457-4cb3-a6db-2ca1890d594a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:32:25.908631 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-de459b6a-4457-4cb3-a6db-2ca1890d594a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 15 08:32:25.908631 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:25.908631 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:25.908631 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:25.908631 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:25.908631 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:25.908631 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:25.908631 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:25.908631 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:25.908631 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:25.908631 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:25.908631 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:25.908631 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:32:25.908631 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:32:25.908631 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:32:25.908631 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:32:25.909533 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:32:25.909533 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:32:25.909533 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:32:25.909533 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:32:25.909533 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:25.909533 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:25.909533 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:25.909533 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:25.909533 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:25.909533 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:25.909533 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:25.909533 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:25.909533 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:25.909533 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:25.909533 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:25.909533 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:25.910358 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:25.910358 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:25.910358 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:25.910358 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:25.910358 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:32:25.910358 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:32:25.910358 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:32:25.910358 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:32:25.910358 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:32:25.910358 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:32:25.910358 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 15 08:32:25.910358 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:25.913062 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 212/430] 10.222.0.30 () {56 vars in 1102 bytes} [Sat Oct 15 08:32:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 08:32:25.914089 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de459b6a-4457-4cb3-a6db-2ca1890d594a None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:25.929776 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:25.933286 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:25.965923 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38e15e3-6555-4d8a-9910-a69efe382b1e None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:26.000819 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a577a055-17a0-4c04-89c5-5f044307fb80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:26.001455 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a577a055-17a0-4c04-89c5-5f044307fb80 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:26.001770 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a577a055-17a0-4c04-89c5-5f044307fb80 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:26.002053 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a577a055-17a0-4c04-89c5-5f044307fb80 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:26.013233 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a577a055-17a0-4c04-89c5-5f044307fb80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:32:26.018875 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-a577a055-17a0-4c04-89c5-5f044307fb80 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 15 08:32:26.018875 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:26.018875 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:26.018875 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:26.018875 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:26.018875 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:26.018875 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:26.018875 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:26.018875 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:26.018875 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:26.018875 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:26.018875 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:26.018875 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:32:26.018875 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:32:26.018875 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:32:26.018875 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:32:26.019811 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:32:26.019811 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:32:26.019811 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:32:26.019811 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:32:26.019811 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:26.019811 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:26.019811 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:26.019811 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:26.019811 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:26.019811 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:26.019811 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:26.019811 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:26.019811 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:26.019811 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:26.019811 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:26.019811 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:26.020590 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:26.020590 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:26.020590 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:26.020590 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:26.020590 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:32:26.020590 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:32:26.020590 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:32:26.020590 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:32:26.020590 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:32:26.020590 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:32:26.020590 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 15 08:32:26.020590 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:26.020590 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 219/431] 10.222.0.30 () {56 vars in 1102 bytes} [Sat Oct 15 08:32:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:26.021240 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a577a055-17a0-4c04-89c5-5f044307fb80 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:26.043213 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de459b6a-4457-4cb3-a6db-2ca1890d594a None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:26.090384 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3cd1fc-b57e-48fc-8805-18ad70e3fd24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:26.091051 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3cd1fc-b57e-48fc-8805-18ad70e3fd24 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:26.091295 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3cd1fc-b57e-48fc-8805-18ad70e3fd24 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:26.091598 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3cd1fc-b57e-48fc-8805-18ad70e3fd24 None admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:26.092849 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3cd1fc-b57e-48fc-8805-18ad70e3fd24 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:26.093168 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3cd1fc-b57e-48fc-8805-18ad70e3fd24 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:26.095111 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3cd1fc-b57e-48fc-8805-18ad70e3fd24 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:26.118519 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 213/432] 10.222.0.30 () {56 vars in 1133 bytes} [Sat Oct 15 08:32:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:26.119496 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3cd1fc-b57e-48fc-8805-18ad70e3fd24 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:26.145773 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a577a055-17a0-4c04-89c5-5f044307fb80 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:26.174453 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:26.185210 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f622b2-a44b-4e03-886e-73d1a5c496c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:26.186344 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f622b2-a44b-4e03-886e-73d1a5c496c1 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:26.187250 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f622b2-a44b-4e03-886e-73d1a5c496c1 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:26.187829 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f622b2-a44b-4e03-886e-73d1a5c496c1 None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:26.188771 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f622b2-a44b-4e03-886e-73d1a5c496c1 None admin] RBAC: Adding query filter params (name=service) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:26.189352 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f622b2-a44b-4e03-886e-73d1a5c496c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:26.191027 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f622b2-a44b-4e03-886e-73d1a5c496c1 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:26.200584 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:26.202526 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 220/433] 10.222.0.30 () {56 vars in 1103 bytes} [Sat Oct 15 08:32:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:26.203214 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f622b2-a44b-4e03-886e-73d1a5c496c1 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:26.234535 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3cd1fc-b57e-48fc-8805-18ad70e3fd24 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:26.270475 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d960837a-6470-40a8-be4c-dcabfb9a72da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:26.271076 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d960837a-6470-40a8-be4c-dcabfb9a72da None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:26.271352 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d960837a-6470-40a8-be4c-dcabfb9a72da None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:26.271631 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d960837a-6470-40a8-be4c-dcabfb9a72da None admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:26.272266 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d960837a-6470-40a8-be4c-dcabfb9a72da None admin] RBAC: Adding query filter params (name=service) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:26.272564 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d960837a-6470-40a8-be4c-dcabfb9a72da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:26.273951 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d960837a-6470-40a8-be4c-dcabfb9a72da None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:26.282395 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 214/434] 10.222.0.30 () {56 vars in 1103 bytes} [Sat Oct 15 08:32:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:26.283224 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf153d-00bb-4ff9-8d10-b5126782369e None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:26.283502 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf153d-00bb-4ff9-8d10-b5126782369e None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:26.283769 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf153d-00bb-4ff9-8d10-b5126782369e None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:26.285116 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 215/435] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:32:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:32:26.289780 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bf153d-00bb-4ff9-8d10-b5126782369e None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:26.318577 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f622b2-a44b-4e03-886e-73d1a5c496c1 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:26.357973 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a06dd12-02f5-4b43-a0e0-478996966f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:26.358513 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a06dd12-02f5-4b43-a0e0-478996966f5a None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:26.358794 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a06dd12-02f5-4b43-a0e0-478996966f5a None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:26.359078 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a06dd12-02f5-4b43-a0e0-478996966f5a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:26.405173 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bf153d-00bb-4ff9-8d10-b5126782369e None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:26.436020 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:26.440415 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:26.446578 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a06dd12-02f5-4b43-a0e0-478996966f5a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:26.447368 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf508cf-8395-4f8b-8309-9d9904052878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:26.448002 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf508cf-8395-4f8b-8309-9d9904052878 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:26.448101 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a06dd12-02f5-4b43-a0e0-478996966f5a None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:26.448285 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf508cf-8395-4f8b-8309-9d9904052878 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:26.448563 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf508cf-8395-4f8b-8309-9d9904052878 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:26.459412 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf508cf-8395-4f8b-8309-9d9904052878 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:26.460872 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf508cf-8395-4f8b-8309-9d9904052878 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:26.475307 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-5bf508cf-8395-4f8b-8309-9d9904052878 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:26.475307 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:26.475307 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:26.475307 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:26.475307 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:26.475307 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:26.475307 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:26.475307 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:26.475307 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:26.475307 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:26.475307 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:26.475307 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:26.475307 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 08:32:26.475307 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 08:32:26.475307 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 08:32:26.475307 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 08:32:26.476197 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:26.476197 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:26.476197 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:26.476197 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:26.476197 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:26.476197 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:26.476197 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:26.476197 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:26.476197 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:26.476197 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:26.476197 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:26.476197 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:26.476197 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:26.476197 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:26.476197 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:26.476197 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:26.476197 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 08:32:26.477027 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 08:32:26.477027 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:26.477027 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:26.477027 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 216/436] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:32:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:26.477413 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf508cf-8395-4f8b-8309-9d9904052878 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:26.554454 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 221/437] 10.222.0.30 () {58 vars in 1282 bytes} [Sat Oct 15 08:32:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 08:32:26.556311 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1106104f-16ae-43aa-ae70-3b9f8aa7ab80 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:26.556630 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1106104f-16ae-43aa-ae70-3b9f8aa7ab80 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:26.556984 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1106104f-16ae-43aa-ae70-3b9f8aa7ab80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:26.575166 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-1106104f-16ae-43aa-ae70-3b9f8aa7ab80 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:32:26.576349 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-1106104f-16ae-43aa-ae70-3b9f8aa7ab80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:32:26.582282 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf508cf-8395-4f8b-8309-9d9904052878 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:26.615311 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5c2707-926b-450c-927e-790adf5a4213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:26.615870 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c2707-926b-450c-927e-790adf5a4213 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:26.616150 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c2707-926b-450c-927e-790adf5a4213 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:26.616426 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c2707-926b-450c-927e-790adf5a4213 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:26.627714 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5c2707-926b-450c-927e-790adf5a4213 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:26.629159 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5c2707-926b-450c-927e-790adf5a4213 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:26.642385 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 222/438] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:32:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 08:32:26.643696 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b37e1b-a929-4149-9aa2-b938752eb318 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:26.643798 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-4a5c2707-926b-450c-927e-790adf5a4213 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:26.643798 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:26.643798 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:26.643798 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:26.643798 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:26.643798 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:26.643798 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:26.643798 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:26.643798 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:26.643798 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:26.643798 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:26.643798 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:26.643798 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 08:32:26.643798 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 08:32:26.643798 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 08:32:26.643798 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 08:32:26.644692 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:26.644692 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:26.644692 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:26.644692 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:26.644692 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:26.644692 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:26.644692 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:26.644692 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:26.644692 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:26.644692 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:26.644692 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:26.644692 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:26.644692 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:26.644692 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:26.644692 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:26.644692 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:26.644692 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 08:32:26.645555 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 08:32:26.645555 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:26.645555 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:26.646065 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b37e1b-a929-4149-9aa2-b938752eb318 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:26.646065 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b37e1b-a929-4149-9aa2-b938752eb318 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:26.646373 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 223/439] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:32:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:32:26.647290 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b37e1b-a929-4149-9aa2-b938752eb318 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:26.649420 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 217/440] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:32:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:26.650523 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5c2707-926b-450c-927e-790adf5a4213 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:26.676926 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:26.688544 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:26.759723 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5c2707-926b-450c-927e-790adf5a4213 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:26.768416 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b37e1b-a929-4149-9aa2-b938752eb318 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:26.794580 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c144f295-2c69-48f0-be0b-ba5f8ea3017b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:26.795215 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c144f295-2c69-48f0-be0b-ba5f8ea3017b None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:26.795501 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c144f295-2c69-48f0-be0b-ba5f8ea3017b None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:26.795782 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c144f295-2c69-48f0-be0b-ba5f8ea3017b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:26.806334 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ddebb3-1ffc-4ff6-9568-a8810ea5cc95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:26.807201 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ddebb3-1ffc-4ff6-9568-a8810ea5cc95 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:26.807358 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c144f295-2c69-48f0-be0b-ba5f8ea3017b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:26.807825 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ddebb3-1ffc-4ff6-9568-a8810ea5cc95 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:26.808280 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ddebb3-1ffc-4ff6-9568-a8810ea5cc95 None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:26.808827 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c144f295-2c69-48f0-be0b-ba5f8ea3017b None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:26.809305 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ddebb3-1ffc-4ff6-9568-a8810ea5cc95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:26.809811 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ddebb3-1ffc-4ff6-9568-a8810ea5cc95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:26.811351 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ddebb3-1ffc-4ff6-9568-a8810ea5cc95 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:26.823779 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 224/441] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 08:32:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:26.824578 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ddebb3-1ffc-4ff6-9568-a8810ea5cc95 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:26.825589 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-c144f295-2c69-48f0-be0b-ba5f8ea3017b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:26.825589 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:26.825589 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:26.825589 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:26.825589 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:26.825589 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:26.825589 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:26.825589 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:26.825589 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:26.825589 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:26.825589 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:26.825589 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:26.825589 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 08:32:26.825589 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 08:32:26.825589 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 08:32:26.825589 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 08:32:26.826489 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:26.826489 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:26.826489 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:26.826489 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:26.826489 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:26.826489 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:26.826489 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:26.826489 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:26.826489 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:26.826489 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:26.826489 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:26.826489 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:26.826489 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:26.826489 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:26.826489 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:26.826489 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:26.826489 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 08:32:26.827334 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 08:32:26.827334 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:26.827334 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:26.829042 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 218/442] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:32:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:26.830544 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703dca97-a05c-4ce6-adbd-e83e80d31fbe None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:26.830995 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703dca97-a05c-4ce6-adbd-e83e80d31fbe None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:26.831453 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703dca97-a05c-4ce6-adbd-e83e80d31fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:26.851115 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-703dca97-a05c-4ce6-adbd-e83e80d31fbe None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:32:26.852814 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-703dca97-a05c-4ce6-adbd-e83e80d31fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:32:26.917643 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 219/443] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:32:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 08:32:26.918428 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703dca97-a05c-4ce6-adbd-e83e80d31fbe None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:26.922099 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:26.943581 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ddebb3-1ffc-4ff6-9568-a8810ea5cc95 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:26.943809 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:26.978150 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999632a2-fd18-4109-a100-6cea9b735cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:26.978853 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999632a2-fd18-4109-a100-6cea9b735cd1 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:26.979142 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999632a2-fd18-4109-a100-6cea9b735cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:26.979420 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999632a2-fd18-4109-a100-6cea9b735cd1 None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:26.980081 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999632a2-fd18-4109-a100-6cea9b735cd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:26.980368 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999632a2-fd18-4109-a100-6cea9b735cd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:26.981764 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999632a2-fd18-4109-a100-6cea9b735cd1 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:26.998926 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 225/444] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 08:32:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:26.998926 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999632a2-fd18-4109-a100-6cea9b735cd1 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:27.036970 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703dca97-a05c-4ce6-adbd-e83e80d31fbe None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:27.071682 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30403bbb-128b-4ded-a21a-09038e734cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:27.072305 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30403bbb-128b-4ded-a21a-09038e734cc1 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:27.072585 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30403bbb-128b-4ded-a21a-09038e734cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:27.072856 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30403bbb-128b-4ded-a21a-09038e734cc1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:27.079774 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30403bbb-128b-4ded-a21a-09038e734cc1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:27.081213 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30403bbb-128b-4ded-a21a-09038e734cc1 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:27.094262 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-30403bbb-128b-4ded-a21a-09038e734cc1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 15 08:32:27.094262 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:27.094262 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:27.094262 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:27.094262 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:27.094262 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:27.094262 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:27.094262 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:27.094262 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:27.094262 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:27.094262 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:27.094262 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:27.094262 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 08:32:27.094262 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 08:32:27.094262 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 08:32:27.094262 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 08:32:27.095158 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:27.095158 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:27.095158 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 08:32:27.095158 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 08:32:27.095158 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 08:32:27.095158 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 08:32:27.095158 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:27.095158 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:27.095158 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:27.095158 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:27.095158 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:27.095158 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:27.095158 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:27.095158 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:27.095158 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:27.095158 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:27.095158 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:27.095158 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:27.100595 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:27.100595 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:27.100595 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 08:32:27.100595 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 08:32:27.100595 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 08:32:27.100595 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 08:32:27.100595 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 08:32:27.100595 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 08:32:27.100595 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 15 08:32:27.100595 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:27.100595 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 220/445] 10.222.0.30 () {56 vars in 1096 bytes} [Sat Oct 15 08:32:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:27.100595 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30403bbb-128b-4ded-a21a-09038e734cc1 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:27.109027 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999632a2-fd18-4109-a100-6cea9b735cd1 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:27.146235 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7987b85-9ce9-4dd2-a3df-4505903a5b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:27.146841 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7987b85-9ce9-4dd2-a3df-4505903a5b36 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:27.147115 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7987b85-9ce9-4dd2-a3df-4505903a5b36 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:27.147373 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7987b85-9ce9-4dd2-a3df-4505903a5b36 None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:27.147998 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7987b85-9ce9-4dd2-a3df-4505903a5b36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:27.148304 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7987b85-9ce9-4dd2-a3df-4505903a5b36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:27.150392 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7987b85-9ce9-4dd2-a3df-4505903a5b36 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:27.159814 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:27.162988 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 226/446] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 08:32:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:27.163808 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7987b85-9ce9-4dd2-a3df-4505903a5b36 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:27.185910 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:27.208049 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30403bbb-128b-4ded-a21a-09038e734cc1 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:27.241577 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78abc7e5-9163-46d6-bf68-9586b3b78156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:27.242234 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78abc7e5-9163-46d6-bf68-9586b3b78156 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:27.242518 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78abc7e5-9163-46d6-bf68-9586b3b78156 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:27.242797 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78abc7e5-9163-46d6-bf68-9586b3b78156 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:27.252763 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78abc7e5-9163-46d6-bf68-9586b3b78156 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:32:27.258083 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-78abc7e5-9163-46d6-bf68-9586b3b78156 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 15 08:32:27.258083 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:27.258083 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:27.258083 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:27.258083 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:27.258083 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:27.258083 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:27.258083 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:27.258083 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:27.258083 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:27.258083 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:27.258083 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:27.258083 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:32:27.258083 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:32:27.258083 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:32:27.258083 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:32:27.259083 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:32:27.259083 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:32:27.259083 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:32:27.259083 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:32:27.259083 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:27.259083 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:27.259083 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:27.259083 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:27.259083 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:27.259083 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:27.259083 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:27.259083 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:27.259083 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:27.259083 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:27.259083 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:27.259083 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:27.259960 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:27.259960 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:27.259960 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:27.259960 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:27.259960 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:32:27.259960 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:32:27.259960 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:32:27.259960 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:32:27.259960 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:32:27.259960 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:32:27.259960 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 15 08:32:27.259960 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:27.259960 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 221/447] 10.222.0.30 () {56 vars in 1102 bytes} [Sat Oct 15 08:32:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:27.260601 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78abc7e5-9163-46d6-bf68-9586b3b78156 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:27.266389 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7987b85-9ce9-4dd2-a3df-4505903a5b36 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:27.299413 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a006c63e-a5a5-43c6-ba62-0b3ce9d9e9b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:27.300035 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a006c63e-a5a5-43c6-ba62-0b3ce9d9e9b5 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:27.300319 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a006c63e-a5a5-43c6-ba62-0b3ce9d9e9b5 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:27.300595 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a006c63e-a5a5-43c6-ba62-0b3ce9d9e9b5 None admin] PATH_INFO: `/v3/users/glance` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:27.308315 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a006c63e-a5a5-43c6-ba62-0b3ce9d9e9b5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:27.309777 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a006c63e-a5a5-43c6-ba62-0b3ce9d9e9b5 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:27.320631 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-a006c63e-a5a5-43c6-ba62-0b3ce9d9e9b5 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 15 08:32:27.320631 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:27.320631 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:27.320631 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:27.320631 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:27.320631 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:27.320631 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:27.320631 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:27.320631 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:27.320631 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:27.320631 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:27.320631 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:27.320631 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 08:32:27.320631 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 08:32:27.320631 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 08:32:27.320631 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 08:32:27.322331 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:27.322331 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:27.322331 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 08:32:27.322331 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 08:32:27.322331 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 08:32:27.322331 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 08:32:27.322331 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:27.322331 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:27.322331 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:27.322331 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:27.322331 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:27.322331 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:27.322331 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:27.322331 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:27.322331 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:27.322331 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:27.322331 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:27.322331 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:27.323262 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:27.323262 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:27.323262 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 08:32:27.323262 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 08:32:27.323262 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 08:32:27.323262 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 08:32:27.323262 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 08:32:27.323262 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 08:32:27.323262 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 15 08:32:27.323262 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:27.323262 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 227/448] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 08:32:27 2022] GET /identity/v3/users/glance => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:27.323832 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a006c63e-a5a5-43c6-ba62-0b3ce9d9e9b5 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:27.361167 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78abc7e5-9163-46d6-bf68-9586b3b78156 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:27.383235 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:27.395474 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2fa96b-4aaa-4105-a6f5-5d081c401593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:27.396296 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2fa96b-4aaa-4105-a6f5-5d081c401593 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:27.396596 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2fa96b-4aaa-4105-a6f5-5d081c401593 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:27.396862 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2fa96b-4aaa-4105-a6f5-5d081c401593 None admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:27.397598 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2fa96b-4aaa-4105-a6f5-5d081c401593 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:27.397941 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2fa96b-4aaa-4105-a6f5-5d081c401593 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:27.399429 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2fa96b-4aaa-4105-a6f5-5d081c401593 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:27.408649 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:27.419333 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 222/449] 10.222.0.30 () {56 vars in 1135 bytes} [Sat Oct 15 08:32:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:27.420138 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2fa96b-4aaa-4105-a6f5-5d081c401593 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:27.432973 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a006c63e-a5a5-43c6-ba62-0b3ce9d9e9b5 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:27.468777 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792269d3-c624-49a0-a03d-fda9740c0eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:27.469339 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792269d3-c624-49a0-a03d-fda9740c0eed None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:27.469621 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792269d3-c624-49a0-a03d-fda9740c0eed None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:27.469924 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792269d3-c624-49a0-a03d-fda9740c0eed None admin] PATH_INFO: `/v3/projects/service` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:27.476924 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792269d3-c624-49a0-a03d-fda9740c0eed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:27.478416 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792269d3-c624-49a0-a03d-fda9740c0eed None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:27.489310 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-792269d3-c624-49a0-a03d-fda9740c0eed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 08:32:27.489310 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:27.489310 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:27.489310 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:27.489310 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:27.489310 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:27.489310 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:27.489310 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:27.489310 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:27.489310 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:27.489310 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:27.489310 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:27.489310 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 08:32:27.489310 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 08:32:27.489310 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 08:32:27.490173 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 08:32:27.490173 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:27.490173 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:27.490173 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:27.490173 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:27.490173 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:27.490173 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:27.490173 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:27.490173 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:27.490173 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:27.490173 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:27.490173 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:27.490173 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:27.490173 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:27.490173 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:27.490173 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:27.490173 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:27.491044 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 08:32:27.491044 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 08:32:27.491044 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 08:32:27.491044 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 08:32:27.491044 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 08:32:27.491044 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 08:32:27.491044 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 08:32:27.491044 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:27.491044 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 228/450] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 08:32:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:27.492135 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792269d3-c624-49a0-a03d-fda9740c0eed None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:27.521150 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2fa96b-4aaa-4105-a6f5-5d081c401593 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:27.554223 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdb78c6-cc95-4bde-8f43-f6dbbd6ec991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:27.554857 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdb78c6-cc95-4bde-8f43-f6dbbd6ec991 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:27.555138 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdb78c6-cc95-4bde-8f43-f6dbbd6ec991 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:27.555442 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdb78c6-cc95-4bde-8f43-f6dbbd6ec991 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:27.565283 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdb78c6-cc95-4bde-8f43-f6dbbd6ec991 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:32:27.570236 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-efdb78c6-cc95-4bde-8f43-f6dbbd6ec991 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 15 08:32:27.570236 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:27.570236 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:27.570236 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:27.570236 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:27.570236 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:27.570236 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:27.570236 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:27.570236 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:27.570236 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:27.570236 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:27.570236 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:27.570236 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:32:27.570236 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:32:27.570236 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:32:27.570236 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:32:27.571680 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:32:27.571680 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:32:27.571680 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:32:27.571680 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:32:27.571680 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:27.571680 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:27.571680 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:27.571680 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:27.571680 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:27.571680 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:27.571680 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:27.571680 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:27.571680 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:27.571680 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:27.571680 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:27.571680 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:27.572445 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:27.572445 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:27.572445 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:27.572445 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:27.572445 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:32:27.572445 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:32:27.572445 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:32:27.572445 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:32:27.572445 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:32:27.572445 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:32:27.572445 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 15 08:32:27.572445 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:27.576796 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 223/451] 10.222.0.30 () {56 vars in 1102 bytes} [Sat Oct 15 08:32:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:27.577619 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdb78c6-cc95-4bde-8f43-f6dbbd6ec991 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:27.588005 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792269d3-c624-49a0-a03d-fda9740c0eed None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:27.607343 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:27.621458 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f946e9c-b276-46f1-b0b3-0601cfd39e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:27.622088 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f946e9c-b276-46f1-b0b3-0601cfd39e03 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:27.622369 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f946e9c-b276-46f1-b0b3-0601cfd39e03 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:27.622647 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f946e9c-b276-46f1-b0b3-0601cfd39e03 None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:27.623299 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f946e9c-b276-46f1-b0b3-0601cfd39e03 None admin] RBAC: Adding query filter params (name=service) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:27.623603 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f946e9c-b276-46f1-b0b3-0601cfd39e03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:27.624950 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f946e9c-b276-46f1-b0b3-0601cfd39e03 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:27.632454 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:27.632725 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 229/452] 10.222.0.30 () {56 vars in 1103 bytes} [Sat Oct 15 08:32:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:27.634351 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f946e9c-b276-46f1-b0b3-0601cfd39e03 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:27.680672 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdb78c6-cc95-4bde-8f43-f6dbbd6ec991 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:27.713885 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f19d857-8fa7-4e09-9a9d-38c129d91914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:27.714486 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f19d857-8fa7-4e09-9a9d-38c129d91914 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:27.714768 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f19d857-8fa7-4e09-9a9d-38c129d91914 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:27.715044 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f19d857-8fa7-4e09-9a9d-38c129d91914 None admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:27.715766 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f19d857-8fa7-4e09-9a9d-38c129d91914 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:27.716072 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f19d857-8fa7-4e09-9a9d-38c129d91914 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:27.717512 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f19d857-8fa7-4e09-9a9d-38c129d91914 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:27.732565 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f946e9c-b276-46f1-b0b3-0601cfd39e03 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:27.734258 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 224/453] 10.222.0.30 () {56 vars in 1137 bytes} [Sat Oct 15 08:32:27 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:27.735044 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f19d857-8fa7-4e09-9a9d-38c129d91914 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:27.771121 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5e66be-f41f-4f34-a31e-86626b0ad52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:27.771744 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5e66be-f41f-4f34-a31e-86626b0ad52b None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:27.772029 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5e66be-f41f-4f34-a31e-86626b0ad52b None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:27.772306 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5e66be-f41f-4f34-a31e-86626b0ad52b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:27.782499 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5e66be-f41f-4f34-a31e-86626b0ad52b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:27.783932 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5e66be-f41f-4f34-a31e-86626b0ad52b None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:27.797600 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-ee5e66be-f41f-4f34-a31e-86626b0ad52b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:27.797600 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:27.797600 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:27.797600 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:27.797600 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:27.797600 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:27.797600 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:27.797600 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:27.797600 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:27.797600 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:27.797600 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:27.797600 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:27.797600 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 08:32:27.797600 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 08:32:27.797600 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 08:32:27.797600 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 08:32:27.798669 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:27.798669 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:27.798669 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:27.798669 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:27.798669 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:27.798669 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:27.798669 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:27.798669 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:27.798669 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:27.798669 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:27.798669 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:27.798669 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:27.798669 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:27.798669 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:27.798669 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:27.798669 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:27.798669 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 08:32:27.799711 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 08:32:27.799711 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:27.799711 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:27.799711 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 230/454] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:32:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:27.799968 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5e66be-f41f-4f34-a31e-86626b0ad52b None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:27.834644 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f19d857-8fa7-4e09-9a9d-38c129d91914 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:27.834973 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:27.848232 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:27.868399 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3567584-a03b-4670-b1c5-6d3b995182a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:27.869001 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3567584-a03b-4670-b1c5-6d3b995182a0 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:27.869284 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3567584-a03b-4670-b1c5-6d3b995182a0 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:27.869559 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3567584-a03b-4670-b1c5-6d3b995182a0 None admin] PATH_INFO: `/v3/projects` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:27.870279 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3567584-a03b-4670-b1c5-6d3b995182a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:27.870594 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3567584-a03b-4670-b1c5-6d3b995182a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:27.872031 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3567584-a03b-4670-b1c5-6d3b995182a0 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:27.883166 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 225/455] 10.222.0.30 () {56 vars in 1148 bytes} [Sat Oct 15 08:32:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:27.884536 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3567584-a03b-4670-b1c5-6d3b995182a0 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:27.902050 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5e66be-f41f-4f34-a31e-86626b0ad52b None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:27.934950 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82173fff-2607-45bc-8731-bda9ebc5708a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:27.935541 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82173fff-2607-45bc-8731-bda9ebc5708a None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:27.935821 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82173fff-2607-45bc-8731-bda9ebc5708a None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:27.936090 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82173fff-2607-45bc-8731-bda9ebc5708a None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:27.936739 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82173fff-2607-45bc-8731-bda9ebc5708a None admin] RBAC: Adding query filter params (name=service) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:27.937039 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82173fff-2607-45bc-8731-bda9ebc5708a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:27.938411 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82173fff-2607-45bc-8731-bda9ebc5708a None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:27.946126 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 231/456] 10.222.0.30 () {56 vars in 1103 bytes} [Sat Oct 15 08:32:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:27.946929 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82173fff-2607-45bc-8731-bda9ebc5708a None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:27.983832 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3567584-a03b-4670-b1c5-6d3b995182a0 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:28.016944 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac8334e-75c2-4cee-a76a-74957e56722b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:28.017569 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac8334e-75c2-4cee-a76a-74957e56722b None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:28.017887 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac8334e-75c2-4cee-a76a-74957e56722b None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:28.018171 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac8334e-75c2-4cee-a76a-74957e56722b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:28.028390 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac8334e-75c2-4cee-a76a-74957e56722b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:28.029896 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac8334e-75c2-4cee-a76a-74957e56722b None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:28.040860 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:28.044425 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-0ac8334e-75c2-4cee-a76a-74957e56722b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:28.044425 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:28.044425 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:28.044425 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:28.044425 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:28.044425 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:28.044425 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:28.044425 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:28.044425 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:28.044425 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:28.044425 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:28.044425 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:28.044425 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 08:32:28.044425 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 08:32:28.044425 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 08:32:28.044425 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 08:32:28.045424 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:28.045424 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:28.045424 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:28.045424 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:28.045424 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:28.045424 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:28.045424 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:28.045424 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:28.045424 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:28.045424 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:28.045424 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:28.045424 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:28.045424 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:28.045424 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:28.045424 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:28.045424 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:28.045424 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 08:32:28.046777 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 08:32:28.046777 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:28.046777 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:28.046777 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 226/457] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:32:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:28.046990 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82173fff-2607-45bc-8731-bda9ebc5708a None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:28.050585 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac8334e-75c2-4cee-a76a-74957e56722b None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:28.080018 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:28.080172 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b678630-142e-47ed-adb5-4ca5263ec359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:28.080649 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b678630-142e-47ed-adb5-4ca5263ec359 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:28.080938 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b678630-142e-47ed-adb5-4ca5263ec359 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:28.081216 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b678630-142e-47ed-adb5-4ca5263ec359 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:28.147478 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b678630-142e-47ed-adb5-4ca5263ec359 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:28.148982 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b678630-142e-47ed-adb5-4ca5263ec359 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:28.151133 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac8334e-75c2-4cee-a76a-74957e56722b None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:28.183808 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc589ab1-d033-4837-9d4d-6e9e2ece7641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:28.184402 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc589ab1-d033-4837-9d4d-6e9e2ece7641 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:28.184684 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc589ab1-d033-4837-9d4d-6e9e2ece7641 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:28.184959 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc589ab1-d033-4837-9d4d-6e9e2ece7641 None admin] PATH_INFO: `/v3/domains` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:28.185592 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc589ab1-d033-4837-9d4d-6e9e2ece7641 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:28.185925 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc589ab1-d033-4837-9d4d-6e9e2ece7641 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:28.187289 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc589ab1-d033-4837-9d4d-6e9e2ece7641 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:28.198711 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 227/458] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 08:32:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:28.199498 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc589ab1-d033-4837-9d4d-6e9e2ece7641 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:28.244157 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:28.251463 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 232/459] 10.222.0.30 () {58 vars in 1282 bytes} [Sat Oct 15 08:32:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 08:32:28.252550 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b678630-142e-47ed-adb5-4ca5263ec359 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:28.282990 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:28.298683 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc589ab1-d033-4837-9d4d-6e9e2ece7641 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:28.331513 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2416a4-dd5b-4ee6-8abc-f9622376a147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:28.332111 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2416a4-dd5b-4ee6-8abc-f9622376a147 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:28.332394 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2416a4-dd5b-4ee6-8abc-f9622376a147 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:28.332673 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2416a4-dd5b-4ee6-8abc-f9622376a147 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:28.333384 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2416a4-dd5b-4ee6-8abc-f9622376a147 None admin] RBAC: Adding query filter params (role.id=b1b0b29beb794324826a4c4003ae293b, scope.project.id=5dfceb59b3794439a4586b3cbe408f0e, user.id=af0a8af5cadc4cc1ab73e8eb2a20bb7e) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:28.333727 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2416a4-dd5b-4ee6-8abc-f9622376a147 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:28.335155 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2416a4-dd5b-4ee6-8abc-f9622376a147 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:28.340732 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 228/460] 10.222.0.30 () {56 vars in 1374 bytes} [Sat Oct 15 08:32:28 2022] GET /identity/v3/role_assignments?user.id=af0a8af5cadc4cc1ab73e8eb2a20bb7e&scope.project.id=5dfceb59b3794439a4586b3cbe408f0e&role.id=b1b0b29beb794324826a4c4003ae293b => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:28.341505 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2416a4-dd5b-4ee6-8abc-f9622376a147 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:28.360222 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b678630-142e-47ed-adb5-4ca5263ec359 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:28.407128 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f153858-0955-4864-9fd5-690bf5a5d508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:28.407657 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f153858-0955-4864-9fd5-690bf5a5d508 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:28.407941 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f153858-0955-4864-9fd5-690bf5a5d508 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:28.408217 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f153858-0955-4864-9fd5-690bf5a5d508 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:28.420590 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f153858-0955-4864-9fd5-690bf5a5d508 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:28.422312 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f153858-0955-4864-9fd5-690bf5a5d508 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:28.440339 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-3f153858-0955-4864-9fd5-690bf5a5d508 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:28.440339 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:28.440339 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:28.440339 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:28.440339 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:28.440339 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:28.440339 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:28.440339 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:28.440339 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:28.440339 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:28.440339 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:28.440339 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:28.440339 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 08:32:28.440339 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 08:32:28.440339 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 08:32:28.440339 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 08:32:28.441340 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:28.441340 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:28.441340 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:28.441340 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:28.441340 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:28.441340 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:28.441340 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:28.441340 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:28.441340 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:28.441340 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:28.441340 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:28.441340 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:28.441340 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:28.441340 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:28.441340 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:28.441340 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:28.441340 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 08:32:28.442473 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 08:32:28.442473 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:28.442473 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:28.442473 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 233/461] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:32:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:28.442473 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f153858-0955-4864-9fd5-690bf5a5d508 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:28.465104 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2416a4-dd5b-4ee6-8abc-f9622376a147 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:28.502043 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:28.503031 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bec92de-9d38-4f42-9840-b9a96277d45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:28.503787 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bec92de-9d38-4f42-9840-b9a96277d45f None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:28.504146 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bec92de-9d38-4f42-9840-b9a96277d45f None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:28.504429 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bec92de-9d38-4f42-9840-b9a96277d45f None admin] PATH_INFO: `/v3/domains` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:28.505155 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bec92de-9d38-4f42-9840-b9a96277d45f None admin] RBAC: Adding query filter params (name=Default) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:28.505465 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bec92de-9d38-4f42-9840-b9a96277d45f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:28.506879 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bec92de-9d38-4f42-9840-b9a96277d45f None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:28.521426 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 229/462] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 08:32:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:28.523291 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bec92de-9d38-4f42-9840-b9a96277d45f None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:28.537579 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:28.569878 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f153858-0955-4864-9fd5-690bf5a5d508 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:28.607563 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359cbc62-f7e0-46e9-b166-2386b1cdb166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:28.608425 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359cbc62-f7e0-46e9-b166-2386b1cdb166 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:28.608876 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359cbc62-f7e0-46e9-b166-2386b1cdb166 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:28.609292 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359cbc62-f7e0-46e9-b166-2386b1cdb166 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:28.619367 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359cbc62-f7e0-46e9-b166-2386b1cdb166 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:28.621020 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359cbc62-f7e0-46e9-b166-2386b1cdb166 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:28.632218 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-359cbc62-f7e0-46e9-b166-2386b1cdb166 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 08:32:28.632218 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:28.632218 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:28.632218 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:28.632218 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:28.632218 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:28.632218 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:28.632218 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:28.632218 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:28.632218 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:28.632218 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:28.632218 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:28.632218 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 08:32:28.632218 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 08:32:28.632218 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 08:32:28.633093 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 08:32:28.633093 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:28.633093 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:28.633093 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:28.633093 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:28.633093 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:28.633093 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:28.633093 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:28.633093 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:28.633093 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:28.633093 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:28.633093 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:28.633093 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:28.633093 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:28.633093 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:28.633093 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:28.633093 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:28.635207 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 08:32:28.635207 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 08:32:28.635207 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 08:32:28.635207 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 08:32:28.635207 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 08:32:28.635207 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 08:32:28.635207 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 08:32:28.635207 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:28.635207 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 234/463] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 08:32:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:28.635207 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359cbc62-f7e0-46e9-b166-2386b1cdb166 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:28.644483 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bec92de-9d38-4f42-9840-b9a96277d45f None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:28.686016 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36433a58-b3b6-4734-aa59-df07be2013cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:28.686647 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36433a58-b3b6-4734-aa59-df07be2013cd None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:28.686930 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36433a58-b3b6-4734-aa59-df07be2013cd None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:28.687208 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36433a58-b3b6-4734-aa59-df07be2013cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:28.697543 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36433a58-b3b6-4734-aa59-df07be2013cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:28.699041 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36433a58-b3b6-4734-aa59-df07be2013cd None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:28.714254 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-36433a58-b3b6-4734-aa59-df07be2013cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:28.714254 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:28.714254 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:28.714254 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:28.714254 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:28.714254 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:28.714254 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:28.714254 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:28.714254 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:28.714254 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:28.714254 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:28.714254 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:28.714254 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 08:32:28.714254 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 08:32:28.714254 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 08:32:28.714254 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 08:32:28.715254 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:28.715254 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:28.715254 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:28.715254 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:28.715254 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:28.715254 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:28.715254 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:28.715254 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:28.715254 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:28.715254 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:28.715254 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:28.715254 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:28.715254 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:28.715254 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:28.715254 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:28.715254 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:28.715254 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 08:32:28.716119 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 08:32:28.716119 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:28.716119 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:28.716119 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 230/464] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:32:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:28.716119 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36433a58-b3b6-4734-aa59-df07be2013cd None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:28.748479 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359cbc62-f7e0-46e9-b166-2386b1cdb166 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:28.758648 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:28.784351 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:28.787784 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05b7eb9-e293-4624-9de5-a932f792cc18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:28.788462 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05b7eb9-e293-4624-9de5-a932f792cc18 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:28.788811 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05b7eb9-e293-4624-9de5-a932f792cc18 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:28.789175 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05b7eb9-e293-4624-9de5-a932f792cc18 None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:28.790527 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05b7eb9-e293-4624-9de5-a932f792cc18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:28.790844 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05b7eb9-e293-4624-9de5-a932f792cc18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:28.792219 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05b7eb9-e293-4624-9de5-a932f792cc18 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:28.808530 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 235/465] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 08:32:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:28.808530 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05b7eb9-e293-4624-9de5-a932f792cc18 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:28.832451 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36433a58-b3b6-4734-aa59-df07be2013cd None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:28.867121 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3bd637-f317-4697-9c2a-dcfd4e9631e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:28.868071 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3bd637-f317-4697-9c2a-dcfd4e9631e6 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:28.868361 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3bd637-f317-4697-9c2a-dcfd4e9631e6 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:28.868641 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3bd637-f317-4697-9c2a-dcfd4e9631e6 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:28.877259 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3bd637-f317-4697-9c2a-dcfd4e9631e6 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:28.878751 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3bd637-f317-4697-9c2a-dcfd4e9631e6 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:28.890498 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-1e3bd637-f317-4697-9c2a-dcfd4e9631e6 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 15 08:32:28.890498 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:28.890498 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:28.890498 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:28.890498 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:28.890498 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:28.890498 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:28.890498 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:28.890498 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:28.890498 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:28.890498 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:28.890498 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:28.890498 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 08:32:28.890498 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 08:32:28.890498 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 08:32:28.890498 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 08:32:28.891428 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:28.891428 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:28.891428 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 08:32:28.891428 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 08:32:28.891428 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 08:32:28.891428 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 08:32:28.891428 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:28.891428 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:28.891428 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:28.891428 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:28.891428 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:28.891428 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:28.891428 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:28.891428 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:28.891428 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:28.891428 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:28.891428 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:28.891428 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:28.892570 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:28.892570 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:28.892570 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 08:32:28.892570 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 08:32:28.892570 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 08:32:28.892570 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 08:32:28.892570 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 08:32:28.892570 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 08:32:28.892570 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 15 08:32:28.892570 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:28.897427 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 231/466] 10.222.0.30 () {56 vars in 1102 bytes} [Sat Oct 15 08:32:28 2022] GET /identity/v3/users/neutron => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:28.898315 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3bd637-f317-4697-9c2a-dcfd4e9631e6 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:28.919939 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05b7eb9-e293-4624-9de5-a932f792cc18 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:28.958312 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf8332f-238f-4d4c-84e4-9e63eca284ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:28.958922 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf8332f-238f-4d4c-84e4-9e63eca284ab None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:28.959202 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf8332f-238f-4d4c-84e4-9e63eca284ab None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:28.959479 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf8332f-238f-4d4c-84e4-9e63eca284ab None admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:28.960160 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf8332f-238f-4d4c-84e4-9e63eca284ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:28.960470 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf8332f-238f-4d4c-84e4-9e63eca284ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:28.961973 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf8332f-238f-4d4c-84e4-9e63eca284ab None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:28.973912 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 236/467] 10.222.0.30 () {56 vars in 1148 bytes} [Sat Oct 15 08:32:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:28.974401 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf8332f-238f-4d4c-84e4-9e63eca284ab None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:29.005932 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:29.014744 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3bd637-f317-4697-9c2a-dcfd4e9631e6 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:29.041471 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:29.054494 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff80add-c196-4d35-a30a-3f247dfba696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:29.055108 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff80add-c196-4d35-a30a-3f247dfba696 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:29.055418 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff80add-c196-4d35-a30a-3f247dfba696 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:29.055707 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff80add-c196-4d35-a30a-3f247dfba696 None admin] PATH_INFO: `/v3/domains` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:29.056333 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff80add-c196-4d35-a30a-3f247dfba696 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:29.056653 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff80add-c196-4d35-a30a-3f247dfba696 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:29.058051 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff80add-c196-4d35-a30a-3f247dfba696 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:29.074733 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 232/468] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 08:32:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:29.075511 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff80add-c196-4d35-a30a-3f247dfba696 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:29.081650 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf8332f-238f-4d4c-84e4-9e63eca284ab None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:29.117844 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4984b478-aa1b-4dab-ad97-14077549f519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:29.118829 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4984b478-aa1b-4dab-ad97-14077549f519 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:29.119125 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4984b478-aa1b-4dab-ad97-14077549f519 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:29.119405 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4984b478-aa1b-4dab-ad97-14077549f519 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:29.127333 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4984b478-aa1b-4dab-ad97-14077549f519 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:29.128776 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4984b478-aa1b-4dab-ad97-14077549f519 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:29.140419 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-4984b478-aa1b-4dab-ad97-14077549f519 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 15 08:32:29.140419 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:29.140419 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:29.140419 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:29.140419 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:29.140419 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:29.140419 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:29.140419 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:29.140419 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:29.140419 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:29.140419 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:29.140419 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:29.140419 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 08:32:29.140419 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 08:32:29.140419 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 08:32:29.140419 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 08:32:29.141307 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:29.141307 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:29.141307 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 08:32:29.141307 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 08:32:29.141307 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 08:32:29.141307 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 08:32:29.141307 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:29.141307 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:29.141307 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:29.141307 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:29.141307 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:29.141307 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:29.141307 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:29.141307 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:29.141307 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:29.141307 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:29.141307 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:29.141307 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:29.147105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:29.147105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:29.147105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 08:32:29.147105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 08:32:29.147105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 08:32:29.147105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 08:32:29.147105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 08:32:29.147105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 08:32:29.147105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 15 08:32:29.147105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:29.147105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 237/469] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 08:32:28 2022] GET /identity/v3/users/cinder => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:29.147105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4984b478-aa1b-4dab-ad97-14077549f519 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:29.183776 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff80add-c196-4d35-a30a-3f247dfba696 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:29.219229 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a509fb9d-fb68-4d81-9542-3123e9aa07aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:29.219916 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a509fb9d-fb68-4d81-9542-3123e9aa07aa None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:29.220197 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a509fb9d-fb68-4d81-9542-3123e9aa07aa None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:29.220463 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a509fb9d-fb68-4d81-9542-3123e9aa07aa None admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:29.221172 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a509fb9d-fb68-4d81-9542-3123e9aa07aa None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:29.221481 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a509fb9d-fb68-4d81-9542-3123e9aa07aa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:29.223208 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a509fb9d-fb68-4d81-9542-3123e9aa07aa None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:29.234992 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:29.247784 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 233/470] 10.222.0.30 () {56 vars in 1139 bytes} [Sat Oct 15 08:32:29 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:29.247784 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a509fb9d-fb68-4d81-9542-3123e9aa07aa None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:29.256233 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4984b478-aa1b-4dab-ad97-14077549f519 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:29.289905 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:29.294786 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982098cd-c2eb-4adc-9cbc-c4a0fdf26186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:29.295507 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982098cd-c2eb-4adc-9cbc-c4a0fdf26186 None admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:29.295856 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982098cd-c2eb-4adc-9cbc-c4a0fdf26186 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:29.296195 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982098cd-c2eb-4adc-9cbc-c4a0fdf26186 None admin] PATH_INFO: `/v3/projects/5dfceb59b3794439a4586b3cbe408f0e/users/6e3009e20e4c49e5a74f7fd793b2c329/roles/b1b0b29beb794324826a4c4003ae293b` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:29.311083 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982098cd-c2eb-4adc-9cbc-c4a0fdf26186 None admin] RBAC: Authorizing `identity:create_grant(project_id=5dfceb59b3794439a4586b3cbe408f0e, user_id=6e3009e20e4c49e5a74f7fd793b2c329, role_id=b1b0b29beb794324826a4c4003ae293b)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:29.312582 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982098cd-c2eb-4adc-9cbc-c4a0fdf26186 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:29.329761 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 238/471] 10.222.0.30 () {58 vars in 1439 bytes} [Sat Oct 15 08:32:29 2022] PUT /identity/v3/projects/5dfceb59b3794439a4586b3cbe408f0e/users/6e3009e20e4c49e5a74f7fd793b2c329/roles/b1b0b29beb794324826a4c4003ae293b => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:32:29.330723 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982098cd-c2eb-4adc-9cbc-c4a0fdf26186 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:29.363924 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a509fb9d-fb68-4d81-9542-3123e9aa07aa None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:29.414463 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c2b82b-1c2d-46fe-a7a7-84310b8d05a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:29.416654 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2b82b-1c2d-46fe-a7a7-84310b8d05a1 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:29.417125 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2b82b-1c2d-46fe-a7a7-84310b8d05a1 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:29.417545 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2b82b-1c2d-46fe-a7a7-84310b8d05a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:29.427092 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c2b82b-1c2d-46fe-a7a7-84310b8d05a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:29.429108 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c2b82b-1c2d-46fe-a7a7-84310b8d05a1 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:29.447335 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-37c2b82b-1c2d-46fe-a7a7-84310b8d05a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 08:32:29.447335 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:29.447335 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:29.447335 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:29.447335 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:29.447335 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:29.447335 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:29.447335 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:29.447335 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:29.447335 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:29.447335 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:29.447335 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:29.447335 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 08:32:29.447335 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 08:32:29.447335 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 08:32:29.448453 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 08:32:29.448453 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:29.448453 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:29.448453 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:29.448453 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:29.448453 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:29.448453 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:29.448453 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:29.448453 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:29.448453 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:29.448453 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:29.448453 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:29.448453 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:29.448453 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:29.448453 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:29.448453 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:29.448453 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:29.449469 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 08:32:29.449469 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 08:32:29.449469 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 08:32:29.449469 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 08:32:29.449469 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 08:32:29.449469 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 08:32:29.449469 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 08:32:29.449469 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:29.450979 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 234/472] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 08:32:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:29.452371 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c2b82b-1c2d-46fe-a7a7-84310b8d05a1 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:29.500686 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982098cd-c2eb-4adc-9cbc-c4a0fdf26186 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:29.543151 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905ede8e-8605-4735-8215-376f6aec9701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:29.544062 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905ede8e-8605-4735-8215-376f6aec9701 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:29.544933 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905ede8e-8605-4735-8215-376f6aec9701 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:29.545379 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905ede8e-8605-4735-8215-376f6aec9701 None admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:29.546636 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905ede8e-8605-4735-8215-376f6aec9701 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:29.547105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905ede8e-8605-4735-8215-376f6aec9701 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:29.548858 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905ede8e-8605-4735-8215-376f6aec9701 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:29.559388 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:29.577474 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 239/473] 10.222.0.30 () {56 vars in 1137 bytes} [Sat Oct 15 08:32:29 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:29.578658 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905ede8e-8605-4735-8215-376f6aec9701 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:29.646869 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:29.651902 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c2b82b-1c2d-46fe-a7a7-84310b8d05a1 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:29.698685 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af79eb03-5364-4abe-a4fa-768194bd22eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:29.699457 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af79eb03-5364-4abe-a4fa-768194bd22eb None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:29.700076 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af79eb03-5364-4abe-a4fa-768194bd22eb None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:29.700469 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af79eb03-5364-4abe-a4fa-768194bd22eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:29.717256 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905ede8e-8605-4735-8215-376f6aec9701 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:29.753246 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2368b784-09ec-4031-8d84-c45c7b93a892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:29.754176 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2368b784-09ec-4031-8d84-c45c7b93a892 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:29.754607 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2368b784-09ec-4031-8d84-c45c7b93a892 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:29.755027 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2368b784-09ec-4031-8d84-c45c7b93a892 None admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:29.755871 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2368b784-09ec-4031-8d84-c45c7b93a892 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:29.756323 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2368b784-09ec-4031-8d84-c45c7b93a892 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:29.757996 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2368b784-09ec-4031-8d84-c45c7b93a892 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:29.771776 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 240/474] 10.222.0.30 () {56 vars in 1148 bytes} [Sat Oct 15 08:32:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:29.772940 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2368b784-09ec-4031-8d84-c45c7b93a892 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:29.791479 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af79eb03-5364-4abe-a4fa-768194bd22eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:29.793500 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af79eb03-5364-4abe-a4fa-768194bd22eb None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:29.832988 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:29.878860 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2368b784-09ec-4031-8d84-c45c7b93a892 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:29.900841 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:29.906734 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 235/475] 10.222.0.30 () {58 vars in 1282 bytes} [Sat Oct 15 08:32:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 08:32:29.907805 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af79eb03-5364-4abe-a4fa-768194bd22eb None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:29.913986 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d45399d-e8de-43b9-8770-2de810299760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:29.915185 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d45399d-e8de-43b9-8770-2de810299760 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:29.915478 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d45399d-e8de-43b9-8770-2de810299760 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:29.915747 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d45399d-e8de-43b9-8770-2de810299760 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:29.989203 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d45399d-e8de-43b9-8770-2de810299760 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:29.990857 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d45399d-e8de-43b9-8770-2de810299760 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:30.017908 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af79eb03-5364-4abe-a4fa-768194bd22eb None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:30.050961 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:30.061712 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f00bc-2ad0-449f-8aad-d67366128fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:30.062681 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f00bc-2ad0-449f-8aad-d67366128fc9 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:30.063238 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f00bc-2ad0-449f-8aad-d67366128fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:30.063712 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f00bc-2ad0-449f-8aad-d67366128fc9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:30.064708 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153f00bc-2ad0-449f-8aad-d67366128fc9 None admin] RBAC: Adding query filter params (role.id=b1b0b29beb794324826a4c4003ae293b, scope.project.id=5dfceb59b3794439a4586b3cbe408f0e, user.id=71692d05fd324679b1598feb52b3fe6d) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:30.065218 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153f00bc-2ad0-449f-8aad-d67366128fc9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:30.067077 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153f00bc-2ad0-449f-8aad-d67366128fc9 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:30.073528 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 236/476] 10.222.0.30 () {56 vars in 1374 bytes} [Sat Oct 15 08:32:29 2022] GET /identity/v3/role_assignments?user.id=71692d05fd324679b1598feb52b3fe6d&scope.project.id=5dfceb59b3794439a4586b3cbe408f0e&role.id=b1b0b29beb794324826a4c4003ae293b => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:30.074637 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f00bc-2ad0-449f-8aad-d67366128fc9 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:30.120308 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 241/477] 10.222.0.30 () {58 vars in 1282 bytes} [Sat Oct 15 08:32:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 08:32:30.130403 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d45399d-e8de-43b9-8770-2de810299760 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:30.223193 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:30.293965 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f00bc-2ad0-449f-8aad-d67366128fc9 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:30.312531 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d45399d-e8de-43b9-8770-2de810299760 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:30.345050 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa6dba2-a559-4b27-a635-1671292fd09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:30.345863 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6dba2-a559-4b27-a635-1671292fd09a None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:30.346156 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6dba2-a559-4b27-a635-1671292fd09a None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:30.346427 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6dba2-a559-4b27-a635-1671292fd09a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:30.363290 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa6dba2-a559-4b27-a635-1671292fd09a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:30.363766 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000f5ab3-660b-42bf-91c4-31f075dafde9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:30.364636 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000f5ab3-660b-42bf-91c4-31f075dafde9 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:30.364741 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa6dba2-a559-4b27-a635-1671292fd09a None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:30.365206 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000f5ab3-660b-42bf-91c4-31f075dafde9 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:30.365756 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000f5ab3-660b-42bf-91c4-31f075dafde9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:30.379630 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000f5ab3-660b-42bf-91c4-31f075dafde9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:30.381446 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000f5ab3-660b-42bf-91c4-31f075dafde9 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:30.387457 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-caa6dba2-a559-4b27-a635-1671292fd09a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:30.387457 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:30.387457 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:30.387457 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:30.387457 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:30.387457 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:30.387457 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:30.387457 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:30.387457 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:30.387457 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:30.387457 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:30.387457 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:30.387457 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 08:32:30.387457 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 08:32:30.387457 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 08:32:30.387457 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 08:32:30.388447 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:30.388447 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:30.388447 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:30.388447 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:30.388447 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:30.388447 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:30.388447 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:30.388447 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:30.388447 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:30.388447 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:30.388447 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:30.388447 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:30.388447 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:30.388447 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:30.388447 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:30.388447 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:30.388447 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 08:32:30.389336 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 08:32:30.389336 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:30.389336 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:30.390952 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 237/478] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:32:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:30.403329 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa6dba2-a559-4b27-a635-1671292fd09a None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:30.419854 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-000f5ab3-660b-42bf-91c4-31f075dafde9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:30.419854 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:30.419854 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:30.419854 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:30.419854 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:30.419854 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:30.419854 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:30.419854 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:30.419854 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:30.419854 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:30.419854 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:30.419854 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:30.419854 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 08:32:30.419854 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 08:32:30.419854 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 08:32:30.419854 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 08:32:30.420839 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:30.420839 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:30.420839 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:30.420839 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:30.420839 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:30.420839 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:30.420839 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:30.420839 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:30.420839 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:30.420839 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:30.420839 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:30.420839 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:30.420839 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:30.420839 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:30.420839 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:30.420839 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:30.420839 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 08:32:30.421732 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 08:32:30.421732 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:30.421732 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:30.422835 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 242/479] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:32:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:30.427814 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000f5ab3-660b-42bf-91c4-31f075dafde9 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:30.443989 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:30.525066 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa6dba2-a559-4b27-a635-1671292fd09a None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:30.543776 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:30.552246 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000f5ab3-660b-42bf-91c4-31f075dafde9 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:30.564784 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4678524c-800a-4270-b9a2-fe7d64d09f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:30.565321 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4678524c-800a-4270-b9a2-fe7d64d09f76 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:30.565658 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4678524c-800a-4270-b9a2-fe7d64d09f76 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:30.565908 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4678524c-800a-4270-b9a2-fe7d64d09f76 None admin] PATH_INFO: `/v3/domains` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:30.567235 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4678524c-800a-4270-b9a2-fe7d64d09f76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:30.567235 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4678524c-800a-4270-b9a2-fe7d64d09f76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:30.568232 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4678524c-800a-4270-b9a2-fe7d64d09f76 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:30.582285 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 238/480] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 08:32:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:30.583543 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10ffa3b-f9b2-429a-b01a-97799b7252ef None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:30.583672 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10ffa3b-f9b2-429a-b01a-97799b7252ef None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:30.583893 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10ffa3b-f9b2-429a-b01a-97799b7252ef None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:30.585090 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 239/481] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:32:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:32:30.587675 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10ffa3b-f9b2-429a-b01a-97799b7252ef None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:30.592159 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c750cf0-26cd-4283-90aa-dae011432183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:30.592857 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c750cf0-26cd-4283-90aa-dae011432183 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:30.593193 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c750cf0-26cd-4283-90aa-dae011432183 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:30.593535 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c750cf0-26cd-4283-90aa-dae011432183 None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:30.594266 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c750cf0-26cd-4283-90aa-dae011432183 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:30.594653 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c750cf0-26cd-4283-90aa-dae011432183 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:30.596079 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c750cf0-26cd-4283-90aa-dae011432183 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:30.614727 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 243/482] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 08:32:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:30.616242 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad1fb22-8819-402a-a7d1-d5780d421f99 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:30.616531 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad1fb22-8819-402a-a7d1-d5780d421f99 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:30.616813 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad1fb22-8819-402a-a7d1-d5780d421f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:30.636018 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-8ad1fb22-8819-402a-a7d1-d5780d421f99 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:32:30.637382 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-8ad1fb22-8819-402a-a7d1-d5780d421f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:32:30.704829 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:30.711948 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 244/483] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:32:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 08:32:30.712658 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad1fb22-8819-402a-a7d1-d5780d421f99 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:30.715302 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10ffa3b-f9b2-429a-b01a-97799b7252ef None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:30.784605 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ba17fd-af22-4936-9e9d-7beb452d14b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:30.785515 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba17fd-af22-4936-9e9d-7beb452d14b9 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:30.786021 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba17fd-af22-4936-9e9d-7beb452d14b9 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:30.786520 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba17fd-af22-4936-9e9d-7beb452d14b9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:30.801071 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ba17fd-af22-4936-9e9d-7beb452d14b9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:30.802821 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ba17fd-af22-4936-9e9d-7beb452d14b9 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:30.831742 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-d2ba17fd-af22-4936-9e9d-7beb452d14b9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 08:32:30.831742 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:30.831742 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:30.831742 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:30.831742 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:30.831742 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:30.831742 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:30.831742 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:30.831742 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:30.831742 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:30.831742 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:30.831742 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:30.831742 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 08:32:30.831742 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 08:32:30.831742 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 08:32:30.832687 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 08:32:30.832687 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:30.832687 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:30.832687 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:30.832687 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:30.832687 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:30.832687 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:30.832687 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:30.832687 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:30.832687 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:30.832687 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:30.832687 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:30.832687 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:30.832687 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:30.832687 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:30.832687 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:30.832687 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:30.833536 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 08:32:30.833536 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 08:32:30.833536 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 08:32:30.833536 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 08:32:30.833536 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 08:32:30.833536 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 08:32:30.833536 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 08:32:30.833536 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:30.836183 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 240/484] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 08:32:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:30.843156 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ba17fd-af22-4936-9e9d-7beb452d14b9 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:30.897997 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad1fb22-8819-402a-a7d1-d5780d421f99 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:30.908471 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:30.957013 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094244d5-176f-4cbc-ab4f-bcc20f5a5b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:30.957652 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094244d5-176f-4cbc-ab4f-bcc20f5a5b3e None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:30.957961 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094244d5-176f-4cbc-ab4f-bcc20f5a5b3e None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:30.958242 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094244d5-176f-4cbc-ab4f-bcc20f5a5b3e None admin] PATH_INFO: `/v3/projects/service` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:30.975461 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094244d5-176f-4cbc-ab4f-bcc20f5a5b3e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:30.976911 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094244d5-176f-4cbc-ab4f-bcc20f5a5b3e None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:30.995797 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-094244d5-176f-4cbc-ab4f-bcc20f5a5b3e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 08:32:30.995797 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:30.995797 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:30.995797 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:30.995797 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:30.995797 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:30.995797 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:30.995797 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:30.995797 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:30.995797 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:30.995797 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:30.995797 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:30.995797 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 08:32:30.995797 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 08:32:30.995797 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 08:32:30.996692 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 08:32:30.996692 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:30.996692 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:30.996692 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:30.996692 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:30.996692 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:30.996692 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:30.996692 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:30.996692 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:30.996692 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:30.996692 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:30.996692 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:30.996692 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:30.996692 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:30.996692 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:30.996692 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:30.996692 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:30.998255 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 08:32:30.998255 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 08:32:30.998255 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 08:32:30.998255 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 08:32:30.998255 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 08:32:30.998255 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 08:32:30.998255 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 08:32:30.998255 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:30.998255 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 245/485] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 08:32:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:30.998255 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094244d5-176f-4cbc-ab4f-bcc20f5a5b3e None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:31.017772 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ba17fd-af22-4936-9e9d-7beb452d14b9 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:31.054672 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a6859b-f1bb-429b-a372-b6c4f95d77eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:31.055334 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a6859b-f1bb-429b-a372-b6c4f95d77eb None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:31.055618 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a6859b-f1bb-429b-a372-b6c4f95d77eb None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:31.056036 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a6859b-f1bb-429b-a372-b6c4f95d77eb None admin] PATH_INFO: `/v3/projects` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:31.056675 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a6859b-f1bb-429b-a372-b6c4f95d77eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:31.056986 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a6859b-f1bb-429b-a372-b6c4f95d77eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:31.058565 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a6859b-f1bb-429b-a372-b6c4f95d77eb None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:31.071503 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:31.072046 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 241/486] 10.222.0.30 () {56 vars in 1148 bytes} [Sat Oct 15 08:32:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:31.073393 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a6859b-f1bb-429b-a372-b6c4f95d77eb None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:31.119051 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094244d5-176f-4cbc-ab4f-bcc20f5a5b3e None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:31.156617 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42570332-483f-4b65-b0cb-68ead2743203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:31.158080 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42570332-483f-4b65-b0cb-68ead2743203 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:31.158429 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42570332-483f-4b65-b0cb-68ead2743203 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:31.158771 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42570332-483f-4b65-b0cb-68ead2743203 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:31.171262 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42570332-483f-4b65-b0cb-68ead2743203 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:32:31.177644 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-42570332-483f-4b65-b0cb-68ead2743203 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 08:32:31.177644 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:31.177644 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:31.177644 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:31.177644 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:31.177644 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:31.177644 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:31.177644 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:31.177644 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:31.177644 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:31.177644 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:31.177644 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:31.177644 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:32:31.177644 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:32:31.177644 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:32:31.177644 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:32:31.178530 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:32:31.178530 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:32:31.178530 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:32:31.178530 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:32:31.178530 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:31.178530 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:31.178530 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:31.178530 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:31.178530 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:31.178530 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:31.178530 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:31.178530 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:31.178530 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:31.178530 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:31.178530 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:31.178530 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:31.179334 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:31.179334 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:31.179334 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:31.179334 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:31.179334 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:32:31.179334 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:32:31.179334 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:32:31.179334 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:32:31.179334 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:32:31.179334 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:32:31.179334 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 08:32:31.179334 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:31.179334 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 246/487] 10.222.0.30 () {56 vars in 1096 bytes} [Sat Oct 15 08:32:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:31.180201 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42570332-483f-4b65-b0cb-68ead2743203 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:31.198404 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:31.207361 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a6859b-f1bb-429b-a372-b6c4f95d77eb None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:31.258246 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94411cde-8a33-4a20-94e1-85579318814d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:31.259176 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94411cde-8a33-4a20-94e1-85579318814d None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:31.259878 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94411cde-8a33-4a20-94e1-85579318814d None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:31.260298 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94411cde-8a33-4a20-94e1-85579318814d None admin] PATH_INFO: `/v3/projects` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:31.261764 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94411cde-8a33-4a20-94e1-85579318814d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:31.262361 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94411cde-8a33-4a20-94e1-85579318814d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:31.264338 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94411cde-8a33-4a20-94e1-85579318814d None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:31.285190 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 242/488] 10.222.0.30 () {56 vars in 1148 bytes} [Sat Oct 15 08:32:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:31.285190 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94411cde-8a33-4a20-94e1-85579318814d None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:31.314284 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42570332-483f-4b65-b0cb-68ead2743203 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:31.336253 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:31.358889 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3872f9-41ad-4fa3-8951-d8bff91a0eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:31.359993 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3872f9-41ad-4fa3-8951-d8bff91a0eb6 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:31.360582 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3872f9-41ad-4fa3-8951-d8bff91a0eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:31.361058 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3872f9-41ad-4fa3-8951-d8bff91a0eb6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:31.362150 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3872f9-41ad-4fa3-8951-d8bff91a0eb6 None admin] RBAC: Adding query filter params (role.id=b1b0b29beb794324826a4c4003ae293b, scope.project.id=5dfceb59b3794439a4586b3cbe408f0e, user.id=8b11e349e4cb4558a4c75d843effc0cd) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:31.362733 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3872f9-41ad-4fa3-8951-d8bff91a0eb6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:31.364378 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3872f9-41ad-4fa3-8951-d8bff91a0eb6 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:31.371308 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 247/489] 10.222.0.30 () {56 vars in 1374 bytes} [Sat Oct 15 08:32:31 2022] GET /identity/v3/role_assignments?user.id=8b11e349e4cb4558a4c75d843effc0cd&scope.project.id=5dfceb59b3794439a4586b3cbe408f0e&role.id=b1b0b29beb794324826a4c4003ae293b => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:31.372566 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3872f9-41ad-4fa3-8951-d8bff91a0eb6 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:31.407208 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94411cde-8a33-4a20-94e1-85579318814d None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:31.473393 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab707e1-5148-4d80-a416-f09f600ef99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:31.474172 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab707e1-5148-4d80-a416-f09f600ef99b None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:31.474407 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab707e1-5148-4d80-a416-f09f600ef99b None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:31.474685 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab707e1-5148-4d80-a416-f09f600ef99b None admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:31.475355 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab707e1-5148-4d80-a416-f09f600ef99b None admin] RBAC: Adding query filter params (name=admin) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:31.475658 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab707e1-5148-4d80-a416-f09f600ef99b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:31.477103 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab707e1-5148-4d80-a416-f09f600ef99b None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:31.490833 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3872f9-41ad-4fa3-8951-d8bff91a0eb6 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:31.503578 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 243/490] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 08:32:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:31.508061 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab707e1-5148-4d80-a416-f09f600ef99b None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:31.531933 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e528ed-1820-43a5-b52d-a287bac3f5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:31.533031 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e528ed-1820-43a5-b52d-a287bac3f5e4 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:31.533514 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e528ed-1820-43a5-b52d-a287bac3f5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:31.534038 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e528ed-1820-43a5-b52d-a287bac3f5e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:31.534967 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e528ed-1820-43a5-b52d-a287bac3f5e4 None admin] RBAC: Adding query filter params (role.id=b1b0b29beb794324826a4c4003ae293b, scope.project.id=5dfceb59b3794439a4586b3cbe408f0e, user.id=a649c5fedad847fb9e784e779da3a24e) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:31.535474 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e528ed-1820-43a5-b52d-a287bac3f5e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:31.537181 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e528ed-1820-43a5-b52d-a287bac3f5e4 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:31.542090 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:31.545060 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 248/491] 10.222.0.30 () {56 vars in 1374 bytes} [Sat Oct 15 08:32:31 2022] GET /identity/v3/role_assignments?user.id=a649c5fedad847fb9e784e779da3a24e&scope.project.id=5dfceb59b3794439a4586b3cbe408f0e&role.id=b1b0b29beb794324826a4c4003ae293b => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:31.670810 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab707e1-5148-4d80-a416-f09f600ef99b None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:31.713876 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1725ca-fcea-4401-a168-ae26ee22789f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:31.714545 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1725ca-fcea-4401-a168-ae26ee22789f None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:31.714824 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1725ca-fcea-4401-a168-ae26ee22789f None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:31.715099 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1725ca-fcea-4401-a168-ae26ee22789f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:31.729259 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1725ca-fcea-4401-a168-ae26ee22789f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:31.730876 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1725ca-fcea-4401-a168-ae26ee22789f None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:31.749190 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-fd1725ca-fcea-4401-a168-ae26ee22789f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:31.749190 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:31.749190 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:31.749190 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:31.749190 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:31.749190 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:31.749190 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:31.749190 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:31.749190 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:31.749190 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:31.749190 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:31.749190 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:31.749190 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 08:32:31.749190 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 08:32:31.749190 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 08:32:31.749190 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 08:32:31.750201 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:31.750201 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:31.750201 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:31.750201 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:31.750201 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:31.750201 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:31.750201 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:31.750201 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:31.750201 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:31.750201 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:31.750201 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:31.750201 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:31.750201 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:31.750201 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:31.750201 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:31.750201 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:31.750201 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 08:32:31.751578 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 08:32:31.751578 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:31.751578 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:31.751578 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 244/492] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:32:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 08:32:31.756138 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e528ed-1820-43a5-b52d-a287bac3f5e4 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:31.809919 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:31.845138 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2d8c85-9b46-423a-96f4-37b4778523f3 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:31.845796 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2d8c85-9b46-423a-96f4-37b4778523f3 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:31.846329 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2d8c85-9b46-423a-96f4-37b4778523f3 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:31.849552 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 245/493] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:32:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:32:31.854648 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c82fce6-25e2-4385-a89f-939648239f6f None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:31.855247 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c82fce6-25e2-4385-a89f-939648239f6f None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:31.855735 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c82fce6-25e2-4385-a89f-939648239f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:31.866237 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e528ed-1820-43a5-b52d-a287bac3f5e4 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:31.875822 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-6c82fce6-25e2-4385-a89f-939648239f6f None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:32:31.877372 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-6c82fce6-25e2-4385-a89f-939648239f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:32:31.904447 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe47e94-11db-4466-b945-7519267ed675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:31.905060 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe47e94-11db-4466-b945-7519267ed675 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:31.905348 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe47e94-11db-4466-b945-7519267ed675 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:31.905623 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe47e94-11db-4466-b945-7519267ed675 None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:31.906295 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe47e94-11db-4466-b945-7519267ed675 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:31.906624 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe47e94-11db-4466-b945-7519267ed675 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:31.908303 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe47e94-11db-4466-b945-7519267ed675 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:31.931503 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 249/494] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 08:32:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:31.936384 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe47e94-11db-4466-b945-7519267ed675 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:31.955913 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:31.966391 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 246/495] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:32:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 08:32:32.056714 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe47e94-11db-4466-b945-7519267ed675 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:32.064390 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:32.094368 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a31b9b-b337-4ffe-ac2a-3a2676265512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:32.095011 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a31b9b-b337-4ffe-ac2a-3a2676265512 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:32.095354 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a31b9b-b337-4ffe-ac2a-3a2676265512 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:32.095635 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a31b9b-b337-4ffe-ac2a-3a2676265512 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:32.103119 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a31b9b-b337-4ffe-ac2a-3a2676265512 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:32.104557 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a31b9b-b337-4ffe-ac2a-3a2676265512 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:32.116768 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-b8a31b9b-b337-4ffe-ac2a-3a2676265512 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 15 08:32:32.116768 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:32.116768 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:32.116768 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:32.116768 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:32.116768 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:32.116768 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:32.116768 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:32.116768 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:32.116768 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:32.116768 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:32.116768 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:32.116768 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 08:32:32.116768 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 08:32:32.116768 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 08:32:32.116768 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 08:32:32.117836 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:32.117836 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:32.117836 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 08:32:32.117836 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 08:32:32.117836 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 08:32:32.117836 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 08:32:32.117836 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:32.117836 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:32.117836 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:32.117836 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:32.117836 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:32.117836 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:32.117836 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:32.117836 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:32.117836 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:32.117836 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:32.117836 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:32.117836 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:32.119406 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:32.119406 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:32.119406 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 08:32:32.119406 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 08:32:32.119406 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 08:32:32.119406 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 08:32:32.119406 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 08:32:32.119406 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 08:32:32.119406 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 15 08:32:32.119406 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:32.119406 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 250/496] 10.222.0.30 () {56 vars in 1093 bytes} [Sat Oct 15 08:32:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 08:32:32.126280 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c82fce6-25e2-4385-a89f-939648239f6f None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:32.252489 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c82fce6-25e2-4385-a89f-939648239f6f None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:32.299247 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f233fcf8-af74-4a83-ac6f-c92fa463b0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:32.299872 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f233fcf8-af74-4a83-ac6f-c92fa463b0a6 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:32.300158 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f233fcf8-af74-4a83-ac6f-c92fa463b0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:32.300439 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f233fcf8-af74-4a83-ac6f-c92fa463b0a6 None admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:32.301168 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f233fcf8-af74-4a83-ac6f-c92fa463b0a6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:32.301479 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f233fcf8-af74-4a83-ac6f-c92fa463b0a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:32.302959 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f233fcf8-af74-4a83-ac6f-c92fa463b0a6 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:32.328189 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 247/497] 10.222.0.30 () {56 vars in 1133 bytes} [Sat Oct 15 08:32:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:32.333899 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a31b9b-b337-4ffe-ac2a-3a2676265512 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:32.401790 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f233fcf8-af74-4a83-ac6f-c92fa463b0a6 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:32.494154 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a31b9b-b337-4ffe-ac2a-3a2676265512 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:32.496997 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:32.563142 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f68c18-e3f4-4c58-ab26-d76a0281b88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:32.564115 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f68c18-e3f4-4c58-ab26-d76a0281b88a None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:32.564594 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f68c18-e3f4-4c58-ab26-d76a0281b88a None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:32.565080 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f68c18-e3f4-4c58-ab26-d76a0281b88a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:32.577103 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f233fcf8-af74-4a83-ac6f-c92fa463b0a6 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:32.577663 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f68c18-e3f4-4c58-ab26-d76a0281b88a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:32.579362 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f68c18-e3f4-4c58-ab26-d76a0281b88a None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:32.594105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-54f68c18-e3f4-4c58-ab26-d76a0281b88a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:32.594105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:32.594105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:32.594105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:32.594105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:32.594105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:32.594105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:32.594105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:32.594105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:32.594105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:32.594105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:32.594105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:32.594105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 08:32:32.594105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 08:32:32.594105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 08:32:32.594105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 08:32:32.595060 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:32.595060 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:32.595060 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:32.595060 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:32.595060 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:32.595060 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:32.595060 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:32.595060 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:32.595060 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:32.595060 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:32.595060 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:32.595060 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:32.595060 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:32.595060 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:32.595060 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:32.595060 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:32.595060 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 08:32:32.596001 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 08:32:32.596001 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:32.596001 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:32.598325 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 251/498] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:32:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:32.604509 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f68c18-e3f4-4c58-ab26-d76a0281b88a None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:32.639147 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a03a6f2-8979-4056-bf21-f75468ede917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:32.640163 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03a6f2-8979-4056-bf21-f75468ede917 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:32.640637 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03a6f2-8979-4056-bf21-f75468ede917 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:32.641152 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03a6f2-8979-4056-bf21-f75468ede917 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:32.646983 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:32.653358 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a03a6f2-8979-4056-bf21-f75468ede917 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:32:32.658644 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-7a03a6f2-8979-4056-bf21-f75468ede917 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 15 08:32:32.658644 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:32.658644 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:32.658644 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:32.658644 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:32.658644 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:32.658644 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:32.658644 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:32.658644 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:32.658644 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:32.658644 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:32.658644 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:32.658644 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:32:32.658644 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:32:32.658644 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:32:32.658644 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:32:32.659554 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:32:32.659554 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:32:32.659554 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:32:32.659554 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:32:32.659554 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:32.659554 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:32.659554 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:32.659554 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:32.659554 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:32.659554 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:32.659554 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:32.659554 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:32.659554 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:32.659554 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:32.659554 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:32.659554 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:32.660375 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:32.660375 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:32.660375 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:32.660375 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:32.660375 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:32:32.660375 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:32:32.660375 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:32:32.660375 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:32:32.660375 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:32:32.660375 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:32:32.660375 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 15 08:32:32.660375 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:32.668210 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 248/499] 10.222.0.30 () {56 vars in 1102 bytes} [Sat Oct 15 08:32:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 08:32:32.671089 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a03a6f2-8979-4056-bf21-f75468ede917 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:32.742270 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f68c18-e3f4-4c58-ab26-d76a0281b88a None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:32.793177 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc98993-1f42-4efc-9e7e-8f04b7900d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:32.794412 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc98993-1f42-4efc-9e7e-8f04b7900d40 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:32.796149 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc98993-1f42-4efc-9e7e-8f04b7900d40 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:32.796625 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc98993-1f42-4efc-9e7e-8f04b7900d40 None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:32.797488 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc98993-1f42-4efc-9e7e-8f04b7900d40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:32.798034 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc98993-1f42-4efc-9e7e-8f04b7900d40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:32.802939 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc98993-1f42-4efc-9e7e-8f04b7900d40 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:32.818664 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 252/500] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 08:32:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:32.821390 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab3c2d4-a03d-43d5-b34f-1d4e64409e2a None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:32.821926 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab3c2d4-a03d-43d5-b34f-1d4e64409e2a None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:32.822411 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab3c2d4-a03d-43d5-b34f-1d4e64409e2a None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:32.824245 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 253/501] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:32:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:32:32.825513 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab3c2d4-a03d-43d5-b34f-1d4e64409e2a None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:32.841820 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a03a6f2-8979-4056-bf21-f75468ede917 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:32.863234 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:32.888906 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdde69ec-5a67-4395-b6de-68ca8d5e29d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:32.889742 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde69ec-5a67-4395-b6de-68ca8d5e29d0 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:32.890216 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde69ec-5a67-4395-b6de-68ca8d5e29d0 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:32.890630 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde69ec-5a67-4395-b6de-68ca8d5e29d0 None admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:32.891485 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdde69ec-5a67-4395-b6de-68ca8d5e29d0 None admin] RBAC: Adding query filter params (name=service) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:32.892717 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdde69ec-5a67-4395-b6de-68ca8d5e29d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:32.894850 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdde69ec-5a67-4395-b6de-68ca8d5e29d0 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:32.911598 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 249/502] 10.222.0.30 () {56 vars in 1103 bytes} [Sat Oct 15 08:32:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 08:32:32.911598 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998da4e-c548-4ae3-94dd-4d3f99c085c8 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:32.911598 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998da4e-c548-4ae3-94dd-4d3f99c085c8 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:32.911598 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998da4e-c548-4ae3-94dd-4d3f99c085c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:32.927295 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:32.930770 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-2998da4e-c548-4ae3-94dd-4d3f99c085c8 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:32:32.932448 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-2998da4e-c548-4ae3-94dd-4d3f99c085c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:32:32.959134 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab3c2d4-a03d-43d5-b34f-1d4e64409e2a None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:33.002603 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 250/503] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:32:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 08:32:33.003543 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2998da4e-c548-4ae3-94dd-4d3f99c085c8 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:33.017701 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15b7d39-a99b-4f32-ac26-eb7668555fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:33.017701 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b7d39-a99b-4f32-ac26-eb7668555fbb None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:33.017701 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b7d39-a99b-4f32-ac26-eb7668555fbb None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:33.017701 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b7d39-a99b-4f32-ac26-eb7668555fbb None admin] PATH_INFO: `/v3/projects/service` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:33.032092 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15b7d39-a99b-4f32-ac26-eb7668555fbb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:33.033540 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15b7d39-a99b-4f32-ac26-eb7668555fbb None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:33.054674 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-f15b7d39-a99b-4f32-ac26-eb7668555fbb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 08:32:33.054674 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:33.054674 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:33.054674 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:33.054674 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:33.054674 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:33.054674 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:33.054674 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:33.054674 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:33.054674 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:33.054674 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:33.054674 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:33.054674 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 08:32:33.054674 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 08:32:33.054674 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 08:32:33.055700 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 08:32:33.055700 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:33.055700 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:33.055700 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:33.055700 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:33.055700 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:33.055700 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:33.055700 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:33.055700 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:33.055700 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:33.055700 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:33.055700 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:33.055700 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:33.055700 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:33.055700 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:33.055700 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:33.055700 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:33.057276 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 08:32:33.057276 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 08:32:33.057276 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 08:32:33.057276 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 08:32:33.057276 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 08:32:33.057276 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 08:32:33.057276 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 08:32:33.057276 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:33.057276 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 254/504] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 08:32:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:33.062003 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15b7d39-a99b-4f32-ac26-eb7668555fbb None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:33.176803 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2998da4e-c548-4ae3-94dd-4d3f99c085c8 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:33.201041 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15b7d39-a99b-4f32-ac26-eb7668555fbb None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:33.202011 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:33.247164 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ebd4c4-3a52-4f4a-977b-e4680e6b47ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:33.247794 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ebd4c4-3a52-4f4a-977b-e4680e6b47ee None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:33.248068 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ebd4c4-3a52-4f4a-977b-e4680e6b47ee None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:33.248333 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ebd4c4-3a52-4f4a-977b-e4680e6b47ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:33.268460 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ebd4c4-3a52-4f4a-977b-e4680e6b47ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:33.269944 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ebd4c4-3a52-4f4a-977b-e4680e6b47ee None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:33.270933 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e627ca-f542-4651-8330-73b6404df23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:33.271543 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e627ca-f542-4651-8330-73b6404df23a None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:33.271819 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e627ca-f542-4651-8330-73b6404df23a None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:33.272267 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e627ca-f542-4651-8330-73b6404df23a None admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:33.273289 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e627ca-f542-4651-8330-73b6404df23a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:33.273595 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e627ca-f542-4651-8330-73b6404df23a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:33.275710 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e627ca-f542-4651-8330-73b6404df23a None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:33.284176 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:33.285910 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-02ebd4c4-3a52-4f4a-977b-e4680e6b47ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:33.285910 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:33.285910 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:33.285910 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:33.285910 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:33.285910 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:33.285910 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:33.285910 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:33.285910 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:33.285910 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:33.285910 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:33.285910 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:33.285910 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 08:32:33.285910 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 08:32:33.285910 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 08:32:33.285910 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 08:32:33.287343 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:33.287343 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:33.287343 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:33.287343 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:33.287343 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:33.287343 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:33.287343 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:33.287343 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:33.287343 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:33.287343 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:33.287343 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:33.287343 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:33.287343 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:33.287343 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:33.287343 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:33.287343 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:33.287343 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 08:32:33.293067 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 08:32:33.293067 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:33.293067 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:33.293067 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 251/505] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:32:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:33.293067 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ebd4c4-3a52-4f4a-977b-e4680e6b47ee None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:33.302623 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 255/506] 10.222.0.30 () {56 vars in 1148 bytes} [Sat Oct 15 08:32:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:33.303539 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e627ca-f542-4651-8330-73b6404df23a None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:33.420482 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ebd4c4-3a52-4f4a-977b-e4680e6b47ee None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:33.432536 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e627ca-f542-4651-8330-73b6404df23a None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:33.456515 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f12cc0-97fa-4e6a-8aa3-69e3c1dbe322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:33.457715 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f12cc0-97fa-4e6a-8aa3-69e3c1dbe322 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:33.458308 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f12cc0-97fa-4e6a-8aa3-69e3c1dbe322 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:33.459105 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f12cc0-97fa-4e6a-8aa3-69e3c1dbe322 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:33.471468 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f12cc0-97fa-4e6a-8aa3-69e3c1dbe322 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:33.473099 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f12cc0-97fa-4e6a-8aa3-69e3c1dbe322 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:33.474632 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1372ba13-7902-485f-8ab7-feeaeffb719e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:33.475564 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372ba13-7902-485f-8ab7-feeaeffb719e None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:33.476292 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372ba13-7902-485f-8ab7-feeaeffb719e None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:33.476737 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372ba13-7902-485f-8ab7-feeaeffb719e None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:33.478107 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1372ba13-7902-485f-8ab7-feeaeffb719e None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:33.478555 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1372ba13-7902-485f-8ab7-feeaeffb719e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:33.480045 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1372ba13-7902-485f-8ab7-feeaeffb719e None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:33.489717 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-96f12cc0-97fa-4e6a-8aa3-69e3c1dbe322 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:33.489717 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:33.489717 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:33.489717 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:33.489717 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:33.489717 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:33.489717 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:33.489717 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:33.489717 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:33.489717 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:33.489717 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:33.489717 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:33.489717 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 08:32:33.489717 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 08:32:33.489717 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 08:32:33.489717 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 08:32:33.490885 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:33.490885 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:33.490885 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:33.490885 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:33.490885 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:33.490885 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:33.490885 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:33.490885 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:33.490885 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:33.490885 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:33.490885 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:33.490885 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:33.490885 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:33.490885 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:33.490885 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:33.490885 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:33.490885 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 08:32:33.491973 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 08:32:33.491973 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:33.491973 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:33.492695 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 252/507] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:32:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:33.495511 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 256/508] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 08:32:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:33.498519 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f12cc0-97fa-4e6a-8aa3-69e3c1dbe322 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:33.498701 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1372ba13-7902-485f-8ab7-feeaeffb719e None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:33.516497 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:33.568871 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:33.614692 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1372ba13-7902-485f-8ab7-feeaeffb719e None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:33.621427 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f12cc0-97fa-4e6a-8aa3-69e3c1dbe322 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:33.650474 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074de1b4-3321-4112-825c-27d7ab699820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:33.651514 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074de1b4-3321-4112-825c-27d7ab699820 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:33.651681 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074de1b4-3321-4112-825c-27d7ab699820 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:33.652075 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074de1b4-3321-4112-825c-27d7ab699820 None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:33.652864 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074de1b4-3321-4112-825c-27d7ab699820 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:33.653211 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074de1b4-3321-4112-825c-27d7ab699820 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:33.654845 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074de1b4-3321-4112-825c-27d7ab699820 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:33.660435 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18bfa79-5bee-4234-b2e5-731fe051b43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:33.661289 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18bfa79-5bee-4234-b2e5-731fe051b43d None admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:33.661741 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18bfa79-5bee-4234-b2e5-731fe051b43d None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:33.662158 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18bfa79-5bee-4234-b2e5-731fe051b43d None admin] PATH_INFO: `/v3/projects/5dfceb59b3794439a4586b3cbe408f0e/users/af0a8af5cadc4cc1ab73e8eb2a20bb7e/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:33.664345 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18bfa79-5bee-4234-b2e5-731fe051b43d None admin] RBAC: Authorizing `identity:create_grant(project_id=5dfceb59b3794439a4586b3cbe408f0e, user_id=af0a8af5cadc4cc1ab73e8eb2a20bb7e, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:33.666617 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18bfa79-5bee-4234-b2e5-731fe051b43d None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:33.673807 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 257/509] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 08:32:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:33.675055 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074de1b4-3321-4112-825c-27d7ab699820 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:33.683961 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 253/510] 10.222.0.30 () {58 vars in 1439 bytes} [Sat Oct 15 08:32:33 2022] PUT /identity/v3/projects/5dfceb59b3794439a4586b3cbe408f0e/users/af0a8af5cadc4cc1ab73e8eb2a20bb7e/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:32:33.685226 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18bfa79-5bee-4234-b2e5-731fe051b43d None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:33.777242 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:33.796182 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074de1b4-3321-4112-825c-27d7ab699820 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:33.809717 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18bfa79-5bee-4234-b2e5-731fe051b43d None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:33.836054 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:33.852605 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae939ba-0ba0-4a26-a8e8-837d72f54c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:33.853788 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae939ba-0ba0-4a26-a8e8-837d72f54c3e None admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:33.854359 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae939ba-0ba0-4a26-a8e8-837d72f54c3e None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:33.854831 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae939ba-0ba0-4a26-a8e8-837d72f54c3e None admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:33.855687 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae939ba-0ba0-4a26-a8e8-837d72f54c3e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:33.857794 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae939ba-0ba0-4a26-a8e8-837d72f54c3e None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:33.870949 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d3eeff-e459-46f7-a913-c3262a2430ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:33.874814 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d3eeff-e459-46f7-a913-c3262a2430ca None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:33.875382 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d3eeff-e459-46f7-a913-c3262a2430ca None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:33.875854 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d3eeff-e459-46f7-a913-c3262a2430ca None admin] PATH_INFO: `/v3/users/swift` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:33.890832 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d3eeff-e459-46f7-a913-c3262a2430ca None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:33.893637 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d3eeff-e459-46f7-a913-c3262a2430ca None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:33.907415 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 254/511] 10.222.0.30 () {60 vars in 1132 bytes} [Sat Oct 15 08:32:33 2022] POST /identity/v3/users => generated 254 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:32:33.916172 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-80d3eeff-e459-46f7-a913-c3262a2430ca None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 15 08:32:33.916172 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:33.916172 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:33.916172 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:33.916172 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:33.916172 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:33.916172 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:33.916172 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:33.916172 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:33.916172 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:33.916172 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:33.916172 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:33.916172 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 08:32:33.916172 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 08:32:33.916172 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 08:32:33.916172 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 08:32:33.917500 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:33.917500 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:33.917500 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 08:32:33.917500 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 08:32:33.917500 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 08:32:33.917500 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 08:32:33.917500 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:33.917500 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:33.917500 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:33.917500 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:33.917500 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:33.917500 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:33.917500 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:33.917500 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:33.917500 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:33.917500 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:33.917500 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:33.917500 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:33.918669 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:33.918669 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:33.918669 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 08:32:33.918669 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 08:32:33.918669 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 08:32:33.918669 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 08:32:33.918669 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 08:32:33.918669 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 08:32:33.918669 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 15 08:32:33.918669 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:33.918669 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 258/512] 10.222.0.30 () {56 vars in 1096 bytes} [Sat Oct 15 08:32:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:33.922837 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae939ba-0ba0-4a26-a8e8-837d72f54c3e None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:34.041657 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae939ba-0ba0-4a26-a8e8-837d72f54c3e None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:34.074083 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:34.079160 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7abadeb-a445-4e85-9ab0-74cf50767975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:34.079785 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7abadeb-a445-4e85-9ab0-74cf50767975 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:34.080070 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7abadeb-a445-4e85-9ab0-74cf50767975 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:34.080401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7abadeb-a445-4e85-9ab0-74cf50767975 None admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:34.081150 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7abadeb-a445-4e85-9ab0-74cf50767975 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:34.081462 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7abadeb-a445-4e85-9ab0-74cf50767975 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:34.083046 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7abadeb-a445-4e85-9ab0-74cf50767975 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:34.104918 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 255/513] 10.222.0.30 () {56 vars in 1135 bytes} [Sat Oct 15 08:32:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:34.110645 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d3eeff-e459-46f7-a913-c3262a2430ca None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:34.242519 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d3eeff-e459-46f7-a913-c3262a2430ca None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:34.250588 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47551b55-d951-4f1c-a582-118efe014902 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:34.250876 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47551b55-d951-4f1c-a582-118efe014902 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:34.251157 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47551b55-d951-4f1c-a582-118efe014902 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:34.252809 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 256/514] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:32:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:32:34.258146 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d29e8-bbc1-448a-8236-468831230ac0 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:34.258972 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d29e8-bbc1-448a-8236-468831230ac0 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:34.259247 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d29e8-bbc1-448a-8236-468831230ac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:34.279306 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2732967c-32f8-4440-867d-1fb54728495a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:34.279923 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2732967c-32f8-4440-867d-1fb54728495a None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:34.280199 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2732967c-32f8-4440-867d-1fb54728495a None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:34.280476 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2732967c-32f8-4440-867d-1fb54728495a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:34.281332 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-6b6d29e8-bbc1-448a-8236-468831230ac0 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:32:34.282582 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-6b6d29e8-bbc1-448a-8236-468831230ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:32:34.328415 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:34.353089 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2732967c-32f8-4440-867d-1fb54728495a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:34.354570 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2732967c-32f8-4440-867d-1fb54728495a None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:34.373209 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 257/515] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:32:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 08:32:34.373209 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b07aa-37fb-4193-bbec-180e75e1aceb None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:34.373209 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b07aa-37fb-4193-bbec-180e75e1aceb None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:34.373209 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b07aa-37fb-4193-bbec-180e75e1aceb None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:34.373209 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 258/516] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:32:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:32:34.378685 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c8d66e-7dfe-45a6-96e4-b6479b9b53c3 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:34.379973 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c8d66e-7dfe-45a6-96e4-b6479b9b53c3 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:34.380623 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c8d66e-7dfe-45a6-96e4-b6479b9b53c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:34.419045 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-15c8d66e-7dfe-45a6-96e4-b6479b9b53c3 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:32:34.421352 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-15c8d66e-7dfe-45a6-96e4-b6479b9b53c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:32:34.533755 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 259/517] 10.222.0.30 () {58 vars in 1282 bytes} [Sat Oct 15 08:32:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 08:32:34.539337 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2732967c-32f8-4440-867d-1fb54728495a None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:34.552742 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 259/518] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:32:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 08:32:34.653147 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:34.679350 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2732967c-32f8-4440-867d-1fb54728495a None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:34.688071 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c8d66e-7dfe-45a6-96e4-b6479b9b53c3 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:34.746455 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7525f2-d6d0-45b4-b696-04382c94e7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:34.747101 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7525f2-d6d0-45b4-b696-04382c94e7ac None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:34.747385 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7525f2-d6d0-45b4-b696-04382c94e7ac None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:34.747663 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7525f2-d6d0-45b4-b696-04382c94e7ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:34.748528 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:34.770868 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7525f2-d6d0-45b4-b696-04382c94e7ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:34.772365 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7525f2-d6d0-45b4-b696-04382c94e7ac None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:34.807678 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c8d66e-7dfe-45a6-96e4-b6479b9b53c3 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:34.814125 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-da7525f2-d6d0-45b4-b696-04382c94e7ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:34.814125 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:34.814125 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:34.814125 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:34.814125 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:34.814125 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:34.814125 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:34.814125 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:34.814125 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:34.814125 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:34.814125 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:34.814125 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:34.814125 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 08:32:34.814125 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 08:32:34.814125 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 08:32:34.814125 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 08:32:34.815086 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:34.815086 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:34.815086 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:34.815086 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:34.815086 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:34.815086 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:34.815086 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:34.815086 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:34.815086 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:34.815086 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:34.815086 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:34.815086 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:34.815086 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:34.815086 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:34.815086 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:34.815086 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:34.815086 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 08:32:34.815979 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 08:32:34.815979 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:34.815979 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:34.816539 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 260/519] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:32:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 08:32:34.826803 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7525f2-d6d0-45b4-b696-04382c94e7ac None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:34.877993 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ef5969-c877-4d05-a626-0edcdee73caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:34.878631 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef5969-c877-4d05-a626-0edcdee73caf None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:34.878910 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef5969-c877-4d05-a626-0edcdee73caf None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:34.879184 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef5969-c877-4d05-a626-0edcdee73caf None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:34.879771 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ef5969-c877-4d05-a626-0edcdee73caf None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:34.881151 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ef5969-c877-4d05-a626-0edcdee73caf None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:34.900789 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-50ef5969-c877-4d05-a626-0edcdee73caf None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 15 08:32:34.900789 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:34.900789 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 08:32:34.900789 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 08:32:34.900789 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 08:32:34.900789 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 08:32:34.900789 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 08:32:34.900789 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 08:32:34.900789 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 15 08:32:34.900789 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:34.900789 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 08:32:34.900789 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:34.900789 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:34.900789 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:34.900789 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:34.900789 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:34.900789 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:34.901843 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:34.901843 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:34.901843 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:34.901843 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:34.901843 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:34.901843 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:34.901843 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 08:32:34.901843 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 08:32:34.901843 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 08:32:34.901843 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 08:32:34.901843 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:34.901843 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:34.901843 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:34.901843 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:34.901843 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:34.901843 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:34.902691 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:34.902691 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:34.902691 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:34.902691 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:34.902691 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:34.902691 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:34.902691 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:34.902691 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:34.902691 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:34.902691 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:34.902691 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 08:32:34.902691 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 08:32:34.902691 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 15 08:32:34.902691 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:34.902691 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 260/520] 10.222.0.30 () {56 vars in 1129 bytes} [Sat Oct 15 08:32:34 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 08:32:34.909894 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ef5969-c877-4d05-a626-0edcdee73caf None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:35.000376 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7525f2-d6d0-45b4-b696-04382c94e7ac None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:35.039321 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:35.048952 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3009bf-2896-4a3a-ad06-4d702d9ce5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:35.049648 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3009bf-2896-4a3a-ad06-4d702d9ce5dc None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:35.050039 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3009bf-2896-4a3a-ad06-4d702d9ce5dc None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:35.050945 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3009bf-2896-4a3a-ad06-4d702d9ce5dc None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:35.051675 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3009bf-2896-4a3a-ad06-4d702d9ce5dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:35.052308 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3009bf-2896-4a3a-ad06-4d702d9ce5dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:35.054459 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3009bf-2896-4a3a-ad06-4d702d9ce5dc None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:35.064685 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ef5969-c877-4d05-a626-0edcdee73caf None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:35.070676 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 261/521] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 08:32:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:35.074519 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3009bf-2896-4a3a-ad06-4d702d9ce5dc None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:35.084174 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:35.117715 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd50b9e-c715-4ba0-9da3-42c0259257ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:35.117715 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd50b9e-c715-4ba0-9da3-42c0259257ef None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:35.117715 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd50b9e-c715-4ba0-9da3-42c0259257ef None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:35.117715 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd50b9e-c715-4ba0-9da3-42c0259257ef None admin] PATH_INFO: `/v3/services` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:35.118164 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd50b9e-c715-4ba0-9da3-42c0259257ef None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:35.118164 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd50b9e-c715-4ba0-9da3-42c0259257ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:35.118164 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd50b9e-c715-4ba0-9da3-42c0259257ef None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:35.126979 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 261/522] 10.222.0.30 () {56 vars in 1124 bytes} [Sat Oct 15 08:32:34 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:35.128454 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd50b9e-c715-4ba0-9da3-42c0259257ef None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:35.193308 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3009bf-2896-4a3a-ad06-4d702d9ce5dc None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:35.228670 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9322993d-0742-4a2f-b7f2-f84d3048611b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:35.229300 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322993d-0742-4a2f-b7f2-f84d3048611b None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:35.229637 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322993d-0742-4a2f-b7f2-f84d3048611b None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:35.229945 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322993d-0742-4a2f-b7f2-f84d3048611b None admin] PATH_INFO: `/v3/services/network` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:35.230525 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9322993d-0742-4a2f-b7f2-f84d3048611b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:35.231886 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9322993d-0742-4a2f-b7f2-f84d3048611b None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:35.244808 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-9322993d-0742-4a2f-b7f2-f84d3048611b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 15 08:32:35.244808 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:35.244808 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 08:32:35.244808 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 08:32:35.244808 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 08:32:35.244808 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 08:32:35.244808 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 08:32:35.244808 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 08:32:35.244808 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 15 08:32:35.244808 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:35.244808 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 08:32:35.244808 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:35.244808 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:35.244808 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:35.244808 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:35.244808 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:35.244808 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:35.245848 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:35.245848 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:35.245848 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:35.245848 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:35.245848 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:35.245848 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:35.245848 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 08:32:35.245848 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 08:32:35.245848 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 08:32:35.245848 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 08:32:35.245848 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:35.245848 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:35.245848 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:35.245848 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:35.245848 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:35.245848 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:35.246725 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:35.246725 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:35.246725 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:35.246725 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:35.246725 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:35.246725 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:35.246725 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:35.246725 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:35.246725 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:35.246725 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:35.246725 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 08:32:35.246725 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 08:32:35.246725 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 15 08:32:35.246725 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:35.246725 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 262/523] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 08:32:35 2022] GET /identity/v3/services/network => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:35.247543 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9322993d-0742-4a2f-b7f2-f84d3048611b None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:35.264170 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd50b9e-c715-4ba0-9da3-42c0259257ef None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:35.299164 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e847f-2061-440c-b7ce-7492cf768940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:35.299935 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e847f-2061-440c-b7ce-7492cf768940 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:35.300315 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e847f-2061-440c-b7ce-7492cf768940 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:35.300686 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e847f-2061-440c-b7ce-7492cf768940 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:35.308332 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0e847f-2061-440c-b7ce-7492cf768940 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:35.310020 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0e847f-2061-440c-b7ce-7492cf768940 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:35.318255 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:35.322475 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-0b0e847f-2061-440c-b7ce-7492cf768940 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 08:32:35.322475 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:35.322475 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:35.322475 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:35.322475 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:35.322475 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:35.322475 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:35.322475 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:35.322475 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:35.322475 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:35.322475 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:35.322475 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:35.322475 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 08:32:35.322475 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 08:32:35.322475 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 08:32:35.323509 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 08:32:35.323509 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:35.323509 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:35.323509 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:35.323509 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:35.323509 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:35.323509 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:35.323509 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:35.323509 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:35.323509 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:35.323509 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:35.323509 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:35.323509 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:35.323509 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:35.323509 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:35.323509 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:35.323509 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:35.324415 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 08:32:35.324415 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 08:32:35.324415 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 08:32:35.324415 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 08:32:35.324415 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 08:32:35.324415 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 08:32:35.324415 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 08:32:35.324415 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:35.324829 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 262/524] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 08:32:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:35.325971 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e847f-2061-440c-b7ce-7492cf768940 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:35.363425 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:35.380148 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9322993d-0742-4a2f-b7f2-f84d3048611b None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:35.429408 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2f5115-cc56-4417-9dd7-5d1d6f53dfd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:35.430611 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2f5115-cc56-4417-9dd7-5d1d6f53dfd2 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:35.430901 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2f5115-cc56-4417-9dd7-5d1d6f53dfd2 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:35.431175 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2f5115-cc56-4417-9dd7-5d1d6f53dfd2 None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:35.431896 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2f5115-cc56-4417-9dd7-5d1d6f53dfd2 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:35.432205 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2f5115-cc56-4417-9dd7-5d1d6f53dfd2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:35.433716 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2f5115-cc56-4417-9dd7-5d1d6f53dfd2 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:35.438129 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e847f-2061-440c-b7ce-7492cf768940 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:35.445862 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 263/525] 10.222.0.30 () {56 vars in 1124 bytes} [Sat Oct 15 08:32:35 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:35.445862 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2f5115-cc56-4417-9dd7-5d1d6f53dfd2 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:35.516501 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a27cea-e055-4b4a-8f9d-639b19c5c79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:35.517869 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a27cea-e055-4b4a-8f9d-639b19c5c79d None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:35.518358 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a27cea-e055-4b4a-8f9d-639b19c5c79d None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:35.518846 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a27cea-e055-4b4a-8f9d-639b19c5c79d None admin] PATH_INFO: `/v3/services` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:35.519678 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a27cea-e055-4b4a-8f9d-639b19c5c79d None admin] RBAC: Adding query filter params (name=network) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:35.520195 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a27cea-e055-4b4a-8f9d-639b19c5c79d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:35.521853 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a27cea-e055-4b4a-8f9d-639b19c5c79d None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:35.533776 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 263/526] 10.222.0.30 () {56 vars in 1112 bytes} [Sat Oct 15 08:32:35 2022] GET /identity/v3/services?name=network => generated 124 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:35.547276 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a27cea-e055-4b4a-8f9d-639b19c5c79d None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:35.566646 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2f5115-cc56-4417-9dd7-5d1d6f53dfd2 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:35.586016 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:35.611187 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e15daa-105a-48fe-a13a-927d02d10b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:35.612077 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e15daa-105a-48fe-a13a-927d02d10b4f None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:35.613352 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e15daa-105a-48fe-a13a-927d02d10b4f None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:35.614068 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e15daa-105a-48fe-a13a-927d02d10b4f None admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:35.616271 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e15daa-105a-48fe-a13a-927d02d10b4f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:35.616789 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e15daa-105a-48fe-a13a-927d02d10b4f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:35.619371 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e15daa-105a-48fe-a13a-927d02d10b4f None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:35.650286 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 264/527] 10.222.0.30 () {56 vars in 1148 bytes} [Sat Oct 15 08:32:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:35.655314 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e15daa-105a-48fe-a13a-927d02d10b4f None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:35.749408 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:35.760937 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a27cea-e055-4b4a-8f9d-639b19c5c79d None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:35.831835 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2314a4-8006-4009-b696-03b249e9ec05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:35.832451 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2314a4-8006-4009-b696-03b249e9ec05 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:35.832731 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2314a4-8006-4009-b696-03b249e9ec05 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:35.833007 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2314a4-8006-4009-b696-03b249e9ec05 None admin] PATH_INFO: `/v3/services` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:35.833636 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2314a4-8006-4009-b696-03b249e9ec05 None admin] RBAC: Adding query filter params (type=network) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:35.833963 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2314a4-8006-4009-b696-03b249e9ec05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:35.836550 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2314a4-8006-4009-b696-03b249e9ec05 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:35.850212 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 264/528] 10.222.0.30 () {56 vars in 1112 bytes} [Sat Oct 15 08:32:35 2022] GET /identity/v3/services?type=network => generated 124 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:35.855016 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e15daa-105a-48fe-a13a-927d02d10b4f None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:35.916463 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5e8fa5-fc17-4f6a-8b5e-8383cdd15860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:35.917097 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5e8fa5-fc17-4f6a-8b5e-8383cdd15860 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:35.917380 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5e8fa5-fc17-4f6a-8b5e-8383cdd15860 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:35.917657 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5e8fa5-fc17-4f6a-8b5e-8383cdd15860 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:35.918413 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5e8fa5-fc17-4f6a-8b5e-8383cdd15860 None admin] RBAC: Adding query filter params (role.id=b1b0b29beb794324826a4c4003ae293b, scope.project.id=5dfceb59b3794439a4586b3cbe408f0e, user.id=6e3009e20e4c49e5a74f7fd793b2c329) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:35.918725 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5e8fa5-fc17-4f6a-8b5e-8383cdd15860 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:35.920233 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5e8fa5-fc17-4f6a-8b5e-8383cdd15860 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:35.927290 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 265/529] 10.222.0.30 () {56 vars in 1374 bytes} [Sat Oct 15 08:32:35 2022] GET /identity/v3/role_assignments?user.id=6e3009e20e4c49e5a74f7fd793b2c329&scope.project.id=5dfceb59b3794439a4586b3cbe408f0e&role.id=b1b0b29beb794324826a4c4003ae293b => generated 608 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:36.199205 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baa9f2f-ef5d-4bfb-9454-3e4d3ec04d7a None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:36.199700 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baa9f2f-ef5d-4bfb-9454-3e4d3ec04d7a None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:36.199700 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baa9f2f-ef5d-4bfb-9454-3e4d3ec04d7a None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:36.201013 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 265/530] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:32:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:32:36.206497 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b76e3-c5fd-4930-914f-c2134d64183d None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:36.206793 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b76e3-c5fd-4930-914f-c2134d64183d None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:36.207074 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b76e3-c5fd-4930-914f-c2134d64183d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:36.226022 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-5d8b76e3-c5fd-4930-914f-c2134d64183d None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:32:36.227756 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-5d8b76e3-c5fd-4930-914f-c2134d64183d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:32:36.263651 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:36.297146 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 266/531] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:32:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 08:32:36.448700 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b678ff-f7dd-437a-93c4-5f72fe8eb104 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:36.449050 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b678ff-f7dd-437a-93c4-5f72fe8eb104 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:36.449315 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b678ff-f7dd-437a-93c4-5f72fe8eb104 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:36.450903 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 266/532] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:32:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:32:36.456161 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d788bf02-1be2-418e-8d9b-145fbfb39440 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:36.456456 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d788bf02-1be2-418e-8d9b-145fbfb39440 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:36.456733 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d788bf02-1be2-418e-8d9b-145fbfb39440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:36.485721 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-d788bf02-1be2-418e-8d9b-145fbfb39440 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:32:36.486967 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-d788bf02-1be2-418e-8d9b-145fbfb39440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:32:36.576713 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 267/533] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:32:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 08:32:36.580030 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b678ff-f7dd-437a-93c4-5f72fe8eb104 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:36.756461 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b678ff-f7dd-437a-93c4-5f72fe8eb104 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:36.803674 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e946a4-fef9-4621-bd42-1074d3dad3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:36.804325 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e946a4-fef9-4621-bd42-1074d3dad3a3 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:36.804608 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e946a4-fef9-4621-bd42-1074d3dad3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:36.804946 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e946a4-fef9-4621-bd42-1074d3dad3a3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:36.821110 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e946a4-fef9-4621-bd42-1074d3dad3a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:32:36.826500 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-28e946a4-fef9-4621-bd42-1074d3dad3a3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 08:32:36.826500 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:36.826500 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:36.826500 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:36.826500 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:36.826500 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:36.826500 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:36.826500 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:36.826500 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:36.826500 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:36.826500 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:36.826500 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:36.826500 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:32:36.826500 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:32:36.826500 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:32:36.826500 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:32:36.827401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:32:36.827401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:32:36.827401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:32:36.827401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:32:36.827401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:36.827401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:36.827401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:36.827401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:36.827401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:36.827401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:36.827401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:36.827401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:36.827401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:36.827401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:36.827401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:36.827401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:36.828231 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:36.828231 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:36.828231 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:36.828231 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:36.828231 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:32:36.828231 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:32:36.828231 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:32:36.828231 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:32:36.828231 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:32:36.828231 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:32:36.828231 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 08:32:36.828231 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:36.828231 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 267/534] 10.222.0.30 () {56 vars in 1096 bytes} [Sat Oct 15 08:32:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:36.835023 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d788bf02-1be2-418e-8d9b-145fbfb39440 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:36.981090 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d788bf02-1be2-418e-8d9b-145fbfb39440 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:36.982606 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:36.995618 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e946a4-fef9-4621-bd42-1074d3dad3a3 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:37.022381 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e80e8d-7a1d-495c-a13e-883bba77eb36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:37.023020 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e80e8d-7a1d-495c-a13e-883bba77eb36 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:37.023302 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e80e8d-7a1d-495c-a13e-883bba77eb36 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:37.023573 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e80e8d-7a1d-495c-a13e-883bba77eb36 None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:37.024819 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e80e8d-7a1d-495c-a13e-883bba77eb36 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:37.025121 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e80e8d-7a1d-495c-a13e-883bba77eb36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:37.026933 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e80e8d-7a1d-495c-a13e-883bba77eb36 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:37.036858 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 268/535] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 08:32:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:37.042818 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e80e8d-7a1d-495c-a13e-883bba77eb36 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:37.056906 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:37.195389 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e80e8d-7a1d-495c-a13e-883bba77eb36 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:37.235520 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6cc133-4f5a-41b0-a352-ab411ae5e3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:37.236750 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6cc133-4f5a-41b0-a352-ab411ae5e3f8 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:37.238808 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e946a4-fef9-4621-bd42-1074d3dad3a3 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:37.239654 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6cc133-4f5a-41b0-a352-ab411ae5e3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:37.240133 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6cc133-4f5a-41b0-a352-ab411ae5e3f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:37.257014 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6cc133-4f5a-41b0-a352-ab411ae5e3f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:37.259404 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6cc133-4f5a-41b0-a352-ab411ae5e3f8 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:37.276100 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-2e6cc133-4f5a-41b0-a352-ab411ae5e3f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:37.276100 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:37.276100 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:37.276100 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:37.276100 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:37.276100 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:37.276100 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:37.276100 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:37.276100 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:37.276100 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:37.276100 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:37.276100 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:37.276100 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 08:32:37.276100 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 08:32:37.276100 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 08:32:37.276100 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 08:32:37.277031 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:37.277031 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:37.277031 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:37.277031 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:37.277031 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:37.277031 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:37.277031 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:37.277031 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:37.277031 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:37.277031 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:37.277031 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:37.277031 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:37.277031 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:37.277031 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:37.277031 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:37.277031 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:37.277031 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 08:32:37.277989 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 08:32:37.277989 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:37.277989 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:37.280253 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 269/536] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:32:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:37.286220 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6cc133-4f5a-41b0-a352-ab411ae5e3f8 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:37.306564 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:37.370406 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc9378c-280d-4dc4-9aaf-7a5a8af10cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:37.371058 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc9378c-280d-4dc4-9aaf-7a5a8af10cfd None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:37.371342 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc9378c-280d-4dc4-9aaf-7a5a8af10cfd None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:37.371619 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc9378c-280d-4dc4-9aaf-7a5a8af10cfd None admin] PATH_INFO: `/v3/roles/service` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:37.393641 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6cc133-4f5a-41b0-a352-ab411ae5e3f8 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:37.422328 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc9378c-280d-4dc4-9aaf-7a5a8af10cfd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:32:37.427691 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-cfc9378c-280d-4dc4-9aaf-7a5a8af10cfd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 15 08:32:37.427691 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:37.427691 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:37.427691 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:37.427691 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:37.427691 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:37.427691 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:37.427691 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:37.427691 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:37.427691 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:37.427691 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:37.427691 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:37.427691 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:32:37.427691 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:32:37.427691 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:32:37.427691 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:32:37.428600 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:32:37.428600 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:32:37.428600 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:32:37.428600 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:32:37.428600 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:37.428600 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:37.428600 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:37.428600 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:37.428600 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:37.428600 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:37.428600 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:37.428600 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:37.428600 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:37.428600 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:37.428600 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:37.428600 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:37.429434 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:37.429434 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:37.429434 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:37.429434 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:37.429434 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:32:37.429434 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:32:37.429434 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:32:37.429434 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:32:37.429434 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:32:37.429434 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:32:37.429434 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 15 08:32:37.429434 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:37.429434 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 268/537] 10.222.0.30 () {56 vars in 1102 bytes} [Sat Oct 15 08:32:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 434 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:37.437712 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc9378c-280d-4dc4-9aaf-7a5a8af10cfd None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:37.438438 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e601ee3-3bd6-4279-8a6f-1c7947a28178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:37.439309 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e601ee3-3bd6-4279-8a6f-1c7947a28178 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:37.439791 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e601ee3-3bd6-4279-8a6f-1c7947a28178 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:37.440263 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e601ee3-3bd6-4279-8a6f-1c7947a28178 None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:37.441109 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e601ee3-3bd6-4279-8a6f-1c7947a28178 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:37.441626 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e601ee3-3bd6-4279-8a6f-1c7947a28178 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:37.443246 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e601ee3-3bd6-4279-8a6f-1c7947a28178 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:37.457582 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 270/538] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 08:32:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:37.463443 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e601ee3-3bd6-4279-8a6f-1c7947a28178 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:37.556970 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:37.569471 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e601ee3-3bd6-4279-8a6f-1c7947a28178 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:37.606656 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42355f83-91cf-4215-aa77-69961cbe7930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:37.608179 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42355f83-91cf-4215-aa77-69961cbe7930 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:37.608663 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42355f83-91cf-4215-aa77-69961cbe7930 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:37.609218 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42355f83-91cf-4215-aa77-69961cbe7930 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:37.617181 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42355f83-91cf-4215-aa77-69961cbe7930 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:37.619976 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42355f83-91cf-4215-aa77-69961cbe7930 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:37.632894 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-42355f83-91cf-4215-aa77-69961cbe7930 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 15 08:32:37.632894 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:37.632894 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:37.632894 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:37.632894 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:37.632894 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:37.632894 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:37.632894 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:37.632894 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:37.632894 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:37.632894 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:37.632894 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:37.632894 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 08:32:37.632894 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 08:32:37.632894 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 08:32:37.632894 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 08:32:37.633959 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:37.633959 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:37.633959 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 08:32:37.633959 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 08:32:37.633959 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 08:32:37.633959 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 08:32:37.633959 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:37.633959 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:37.633959 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:37.633959 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:37.633959 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:37.633959 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:37.633959 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:37.633959 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:37.633959 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:37.633959 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:37.633959 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:37.633959 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:37.634901 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:37.634901 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:37.634901 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 08:32:37.634901 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 08:32:37.634901 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 08:32:37.634901 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 08:32:37.634901 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 08:32:37.634901 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 08:32:37.634901 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 15 08:32:37.634901 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:37.637667 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 271/539] 10.222.0.30 () {56 vars in 1093 bytes} [Sat Oct 15 08:32:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:37.643974 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42355f83-91cf-4215-aa77-69961cbe7930 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:37.707102 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:37.754186 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42355f83-91cf-4215-aa77-69961cbe7930 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:37.754594 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc9378c-280d-4dc4-9aaf-7a5a8af10cfd None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:37.791922 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a1d369-e2ca-4050-930d-074aff1f7aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:37.792884 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a1d369-e2ca-4050-930d-074aff1f7aaa None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:37.793367 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a1d369-e2ca-4050-930d-074aff1f7aaa None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:37.793927 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a1d369-e2ca-4050-930d-074aff1f7aaa None admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:37.794856 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a1d369-e2ca-4050-930d-074aff1f7aaa None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:37.795460 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a1d369-e2ca-4050-930d-074aff1f7aaa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:37.797305 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a1d369-e2ca-4050-930d-074aff1f7aaa None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:37.817112 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 272/540] 10.222.0.30 () {56 vars in 1133 bytes} [Sat Oct 15 08:32:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:37.817372 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:37.823476 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a1d369-e2ca-4050-930d-074aff1f7aaa None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:37.858386 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a570243-4bc5-40f7-a058-2cf8f4ff1b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:37.859016 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a570243-4bc5-40f7-a058-2cf8f4ff1b09 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:37.859300 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a570243-4bc5-40f7-a058-2cf8f4ff1b09 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:37.859581 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a570243-4bc5-40f7-a058-2cf8f4ff1b09 None admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:37.860254 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a570243-4bc5-40f7-a058-2cf8f4ff1b09 None admin] RBAC: Adding query filter params (name=service) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:37.860562 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a570243-4bc5-40f7-a058-2cf8f4ff1b09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:37.862049 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a570243-4bc5-40f7-a058-2cf8f4ff1b09 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:37.887782 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 269/541] 10.222.0.30 () {56 vars in 1103 bytes} [Sat Oct 15 08:32:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 450 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:37.906714 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a570243-4bc5-40f7-a058-2cf8f4ff1b09 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:37.935574 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a1d369-e2ca-4050-930d-074aff1f7aaa None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:37.971762 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964077e4-488b-47f1-93e3-8842752b0525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:37.972758 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964077e4-488b-47f1-93e3-8842752b0525 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:37.973243 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964077e4-488b-47f1-93e3-8842752b0525 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:37.973816 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964077e4-488b-47f1-93e3-8842752b0525 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:38.048180 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:38.050774 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964077e4-488b-47f1-93e3-8842752b0525 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:38.052509 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964077e4-488b-47f1-93e3-8842752b0525 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:38.157475 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 273/542] 10.222.0.30 () {58 vars in 1282 bytes} [Sat Oct 15 08:32:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 08:32:38.161710 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a570243-4bc5-40f7-a058-2cf8f4ff1b09 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:38.163116 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86259665-1a06-419a-a0db-291c36e47bb5 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:38.163610 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86259665-1a06-419a-a0db-291c36e47bb5 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:38.164088 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86259665-1a06-419a-a0db-291c36e47bb5 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:38.166278 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 274/543] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:32:38 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:32:38.168354 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86259665-1a06-419a-a0db-291c36e47bb5 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:38.204896 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2755d679-7f77-47aa-9610-88518f4fb2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:38.205525 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2755d679-7f77-47aa-9610-88518f4fb2ee None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:38.205839 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2755d679-7f77-47aa-9610-88518f4fb2ee None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:38.206123 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2755d679-7f77-47aa-9610-88518f4fb2ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:38.219648 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2755d679-7f77-47aa-9610-88518f4fb2ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:38.221902 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2755d679-7f77-47aa-9610-88518f4fb2ee None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:38.230095 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:38.241028 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-2755d679-7f77-47aa-9610-88518f4fb2ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:38.241028 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:38.241028 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:38.241028 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:38.241028 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:38.241028 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:38.241028 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:38.241028 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:38.241028 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:38.241028 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:38.241028 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:38.241028 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:38.241028 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 08:32:38.241028 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 08:32:38.241028 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 08:32:38.241028 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 08:32:38.242987 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:38.242987 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:38.242987 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:38.242987 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:38.242987 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:38.242987 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:38.242987 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:38.242987 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:38.242987 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:38.242987 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:38.242987 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:38.242987 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:38.242987 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:38.242987 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:38.242987 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:38.242987 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:38.242987 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 08:32:38.243885 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 08:32:38.243885 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:38.243885 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:38.245754 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 270/544] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:32:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:38.247208 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eed056c-2f85-447b-889f-3c5a14e1a04e None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:38.248018 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eed056c-2f85-447b-889f-3c5a14e1a04e None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:38.248298 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eed056c-2f85-447b-889f-3c5a14e1a04e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:38.266300 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-2eed056c-2f85-447b-889f-3c5a14e1a04e None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:32:38.267749 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-2eed056c-2f85-447b-889f-3c5a14e1a04e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:32:38.291001 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86259665-1a06-419a-a0db-291c36e47bb5 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:38.294326 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:38.336999 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76fdb70-7524-494c-9933-85496fe7da1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:38.338077 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76fdb70-7524-494c-9933-85496fe7da1b None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:38.338566 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76fdb70-7524-494c-9933-85496fe7da1b None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:38.339108 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76fdb70-7524-494c-9933-85496fe7da1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:38.345796 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 271/545] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:32:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 08:32:38.345796 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eed056c-2f85-447b-889f-3c5a14e1a04e None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:38.364235 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76fdb70-7524-494c-9933-85496fe7da1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:38.365715 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76fdb70-7524-494c-9933-85496fe7da1b None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:38.398309 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-e76fdb70-7524-494c-9933-85496fe7da1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:38.398309 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:38.398309 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:38.398309 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:38.398309 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:38.398309 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:38.398309 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:38.398309 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:38.398309 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:38.398309 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:38.398309 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:38.398309 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:38.398309 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 08:32:38.398309 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 08:32:38.398309 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 08:32:38.398309 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 08:32:38.399963 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:38.399963 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:38.399963 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:38.399963 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:38.399963 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:38.399963 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:38.399963 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:38.399963 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:38.399963 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:38.399963 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:38.399963 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:38.399963 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:38.399963 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:38.399963 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:38.399963 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:38.399963 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:38.399963 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 08:32:38.400893 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 08:32:38.400893 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:38.400893 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:38.406131 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 275/546] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:32:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:38.407044 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76fdb70-7524-494c-9933-85496fe7da1b None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:38.497976 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eed056c-2f85-447b-889f-3c5a14e1a04e None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:38.526060 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:38.534788 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9586e514-134c-4193-b9ef-205cc406b639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:38.535403 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9586e514-134c-4193-b9ef-205cc406b639 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:38.535685 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9586e514-134c-4193-b9ef-205cc406b639 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:38.536031 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9586e514-134c-4193-b9ef-205cc406b639 None admin] PATH_INFO: `/v3/domains` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:38.536599 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9586e514-134c-4193-b9ef-205cc406b639 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:38.536905 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9586e514-134c-4193-b9ef-205cc406b639 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:38.538333 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9586e514-134c-4193-b9ef-205cc406b639 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:38.548355 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76fdb70-7524-494c-9933-85496fe7da1b None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:38.550215 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 272/547] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 08:32:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:38.551762 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fb36ee-b746-4439-bf27-4cfebb72bf7d None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:38.551762 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fb36ee-b746-4439-bf27-4cfebb72bf7d None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:38.551957 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fb36ee-b746-4439-bf27-4cfebb72bf7d None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:38.562039 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 273/548] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:32:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:32:38.562039 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fb36ee-b746-4439-bf27-4cfebb72bf7d None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:38.595470 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0da265-0d54-4c6f-9e05-f819a8f0b655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:38.596355 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0da265-0d54-4c6f-9e05-f819a8f0b655 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:38.597503 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0da265-0d54-4c6f-9e05-f819a8f0b655 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:38.598058 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0da265-0d54-4c6f-9e05-f819a8f0b655 None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:38.599200 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0da265-0d54-4c6f-9e05-f819a8f0b655 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:38.599661 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0da265-0d54-4c6f-9e05-f819a8f0b655 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:38.601395 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0da265-0d54-4c6f-9e05-f819a8f0b655 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:38.620062 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 276/549] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 08:32:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:38.622073 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5649a-baee-4990-82ba-7e6390214f3a None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:38.622528 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5649a-baee-4990-82ba-7e6390214f3a None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:38.622946 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5649a-baee-4990-82ba-7e6390214f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:38.639059 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:38.646470 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-b2b5649a-baee-4990-82ba-7e6390214f3a None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:32:38.647953 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-b2b5649a-baee-4990-82ba-7e6390214f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:32:38.694059 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fb36ee-b746-4439-bf27-4cfebb72bf7d None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:38.714498 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 277/550] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:32:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 08:32:38.715354 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b5649a-baee-4990-82ba-7e6390214f3a None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:38.730418 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455ca4bf-4733-4283-b762-46b1f09f46ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:38.731068 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ca4bf-4733-4283-b762-46b1f09f46ce None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:38.731346 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ca4bf-4733-4283-b762-46b1f09f46ce None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:38.731616 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ca4bf-4733-4283-b762-46b1f09f46ce None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:38.739162 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455ca4bf-4733-4283-b762-46b1f09f46ce None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:38.740605 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455ca4bf-4733-4283-b762-46b1f09f46ce None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:38.752586 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-455ca4bf-4733-4283-b762-46b1f09f46ce None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 15 08:32:38.752586 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:38.752586 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:38.752586 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:38.752586 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:38.752586 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:38.752586 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:38.752586 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:38.752586 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:38.752586 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:38.752586 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:38.752586 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:38.752586 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 08:32:38.752586 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 08:32:38.752586 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 08:32:38.752586 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 08:32:38.753586 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:38.753586 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:38.753586 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 08:32:38.753586 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 08:32:38.753586 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 08:32:38.753586 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 08:32:38.753586 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:38.753586 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:38.753586 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:38.753586 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:38.753586 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:38.753586 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:38.753586 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:38.753586 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:38.753586 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:38.753586 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:38.753586 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:38.753586 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:38.754857 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:38.754857 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:38.754857 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 08:32:38.754857 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 08:32:38.754857 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 08:32:38.754857 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 08:32:38.754857 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 08:32:38.754857 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 08:32:38.754857 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 15 08:32:38.754857 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:38.754857 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 274/551] 10.222.0.30 () {56 vars in 1117 bytes} [Sat Oct 15 08:32:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:38.754857 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455ca4bf-4733-4283-b762-46b1f09f46ce None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:38.803442 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:38.822734 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b5649a-baee-4990-82ba-7e6390214f3a None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:38.859138 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455ca4bf-4733-4283-b762-46b1f09f46ce None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:38.866220 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf7ee25-4ee0-4d3c-be5f-4f1a42be7b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:38.866856 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf7ee25-4ee0-4d3c-be5f-4f1a42be7b20 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:38.867137 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf7ee25-4ee0-4d3c-be5f-4f1a42be7b20 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:38.867413 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf7ee25-4ee0-4d3c-be5f-4f1a42be7b20 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:38.870934 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:38.876506 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf7ee25-4ee0-4d3c-be5f-4f1a42be7b20 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:38.878650 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf7ee25-4ee0-4d3c-be5f-4f1a42be7b20 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:38.890884 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-aaf7ee25-4ee0-4d3c-be5f-4f1a42be7b20 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 08:32:38.890884 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:38.890884 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:38.890884 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:38.890884 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:38.890884 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:38.890884 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:38.890884 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:38.890884 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:38.890884 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:38.890884 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:38.890884 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:38.890884 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 08:32:38.890884 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 08:32:38.890884 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 08:32:38.892029 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 08:32:38.892029 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:38.892029 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:38.892029 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:38.892029 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:38.892029 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:38.892029 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:38.892029 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:38.892029 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:38.892029 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:38.892029 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:38.892029 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:38.892029 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:38.892029 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:38.892029 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:38.892029 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:38.892029 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:38.892927 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 08:32:38.892927 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 08:32:38.892927 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 08:32:38.892927 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 08:32:38.892927 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 08:32:38.892927 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 08:32:38.892927 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 08:32:38.892927 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:38.892927 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 278/552] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 08:32:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:38.893437 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621704dd-ba72-440e-8231-a2f3a85083e5 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:38.894104 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621704dd-ba72-440e-8231-a2f3a85083e5 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:38.894104 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621704dd-ba72-440e-8231-a2f3a85083e5 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:38.897823 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 279/553] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:32:38 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:32:38.897823 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621704dd-ba72-440e-8231-a2f3a85083e5 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:38.901709 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f53029-db07-42dd-be2e-326ef4b9619c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:38.902331 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f53029-db07-42dd-be2e-326ef4b9619c None admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:38.902613 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f53029-db07-42dd-be2e-326ef4b9619c None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:38.902882 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f53029-db07-42dd-be2e-326ef4b9619c None admin] PATH_INFO: `/v3/services` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:38.903546 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f53029-db07-42dd-be2e-326ef4b9619c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:38.904939 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f53029-db07-42dd-be2e-326ef4b9619c None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:38.922790 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 275/554] 10.222.0.30 () {60 vars in 1141 bytes} [Sat Oct 15 08:32:38 2022] POST /identity/v3/services => generated 251 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:32:38.923444 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f53029-db07-42dd-be2e-326ef4b9619c None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:39.041539 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621704dd-ba72-440e-8231-a2f3a85083e5 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:39.058313 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f53029-db07-42dd-be2e-326ef4b9619c None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:39.070098 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:39.089810 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd4d9fc-d8da-4d39-affc-d7b4fba6742b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:39.090549 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd4d9fc-d8da-4d39-affc-d7b4fba6742b None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:39.090835 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd4d9fc-d8da-4d39-affc-d7b4fba6742b None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:39.091111 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd4d9fc-d8da-4d39-affc-d7b4fba6742b None admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:39.091849 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd4d9fc-d8da-4d39-affc-d7b4fba6742b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:39.092159 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd4d9fc-d8da-4d39-affc-d7b4fba6742b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:39.093643 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd4d9fc-d8da-4d39-affc-d7b4fba6742b None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:39.100428 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c108db-02ae-4f49-a07c-32472057c1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:39.100428 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c108db-02ae-4f49-a07c-32472057c1d8 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:39.100428 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c108db-02ae-4f49-a07c-32472057c1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:39.100728 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c108db-02ae-4f49-a07c-32472057c1d8 None admin] PATH_INFO: `/v3/projects` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:39.101363 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c108db-02ae-4f49-a07c-32472057c1d8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:39.101700 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c108db-02ae-4f49-a07c-32472057c1d8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:39.103267 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c108db-02ae-4f49-a07c-32472057c1d8 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:39.115697 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 276/556] 10.222.0.30 () {56 vars in 1148 bytes} [Sat Oct 15 08:32:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:39.116940 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570644d-c4a8-4f6e-a39d-212e62a12e5c None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:39.117081 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570644d-c4a8-4f6e-a39d-212e62a12e5c None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:39.117359 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570644d-c4a8-4f6e-a39d-212e62a12e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:39.120619 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 280/556] 10.222.0.30 () {56 vars in 1149 bytes} [Sat Oct 15 08:32:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:39.121557 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd4d9fc-d8da-4d39-affc-d7b4fba6742b None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:39.138181 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-7570644d-c4a8-4f6e-a39d-212e62a12e5c None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:32:39.139379 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-7570644d-c4a8-4f6e-a39d-212e62a12e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:32:39.173368 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:39.216172 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 277/557] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:32:39 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 08:32:39.217377 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7570644d-c4a8-4f6e-a39d-212e62a12e5c None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:39.231252 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd4d9fc-d8da-4d39-affc-d7b4fba6742b None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:39.269462 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bfb374-7fb8-4601-8eb4-3d602087bc57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:39.270518 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bfb374-7fb8-4601-8eb4-3d602087bc57 None admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:39.270985 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bfb374-7fb8-4601-8eb4-3d602087bc57 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:39.271438 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bfb374-7fb8-4601-8eb4-3d602087bc57 None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:39.272454 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bfb374-7fb8-4601-8eb4-3d602087bc57 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:39.274562 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bfb374-7fb8-4601-8eb4-3d602087bc57 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:39.295471 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 281/558] 10.222.0.30 () {60 vars in 1141 bytes} [Sat Oct 15 08:32:39 2022] POST /identity/v3/services => generated 240 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:32:39.303353 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bfb374-7fb8-4601-8eb4-3d602087bc57 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:39.373182 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:39.379437 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7570644d-c4a8-4f6e-a39d-212e62a12e5c None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:39.450562 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083098a4-62b9-4e54-86fb-dfd341323335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:39.451206 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083098a4-62b9-4e54-86fb-dfd341323335 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:39.451488 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083098a4-62b9-4e54-86fb-dfd341323335 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:39.451766 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083098a4-62b9-4e54-86fb-dfd341323335 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:39.464179 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083098a4-62b9-4e54-86fb-dfd341323335 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:39.465907 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083098a4-62b9-4e54-86fb-dfd341323335 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:39.484262 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-083098a4-62b9-4e54-86fb-dfd341323335 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:39.484262 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:39.484262 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:39.484262 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:39.484262 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:39.484262 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:39.484262 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:39.484262 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:39.484262 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:39.484262 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:39.484262 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:39.484262 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:39.484262 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 08:32:39.484262 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 08:32:39.484262 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 08:32:39.484262 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 08:32:39.485365 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:39.485365 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:39.485365 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:39.485365 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:39.485365 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:39.485365 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:39.485365 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:39.485365 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:39.485365 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:39.485365 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:39.485365 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:39.485365 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:39.485365 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:39.485365 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:39.485365 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:39.485365 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:39.485365 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 08:32:39.486283 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 08:32:39.486283 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:39.486283 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:39.486283 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 278/559] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:32:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:39.493697 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083098a4-62b9-4e54-86fb-dfd341323335 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:39.543423 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bfb374-7fb8-4601-8eb4-3d602087bc57 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:39.557196 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:39.585173 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6b1c61-6c73-4b17-960a-c80321dfb003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:39.586420 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6b1c61-6c73-4b17-960a-c80321dfb003 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:39.586707 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6b1c61-6c73-4b17-960a-c80321dfb003 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:39.587247 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6b1c61-6c73-4b17-960a-c80321dfb003 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:39.587979 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6b1c61-6c73-4b17-960a-c80321dfb003 None admin] RBAC: Adding query filter params (role.id=14615e30ed3f4d8cb78ee5e0bf9b5761, scope.project.id=5dfceb59b3794439a4586b3cbe408f0e, user.id=af0a8af5cadc4cc1ab73e8eb2a20bb7e) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:39.588402 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6b1c61-6c73-4b17-960a-c80321dfb003 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:39.590695 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6b1c61-6c73-4b17-960a-c80321dfb003 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:39.597990 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 282/560] 10.222.0.30 () {56 vars in 1374 bytes} [Sat Oct 15 08:32:39 2022] GET /identity/v3/role_assignments?user.id=af0a8af5cadc4cc1ab73e8eb2a20bb7e&scope.project.id=5dfceb59b3794439a4586b3cbe408f0e&role.id=14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 608 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:39.598926 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6b1c61-6c73-4b17-960a-c80321dfb003 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:39.606762 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083098a4-62b9-4e54-86fb-dfd341323335 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:39.656849 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253d1321-79f0-43a0-89fe-9f21719beb3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:39.658033 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253d1321-79f0-43a0-89fe-9f21719beb3c None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:39.658384 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253d1321-79f0-43a0-89fe-9f21719beb3c None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:39.658666 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253d1321-79f0-43a0-89fe-9f21719beb3c None admin] PATH_INFO: `/v3/domains` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:39.659952 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253d1321-79f0-43a0-89fe-9f21719beb3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:39.660402 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253d1321-79f0-43a0-89fe-9f21719beb3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:39.662068 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253d1321-79f0-43a0-89fe-9f21719beb3c None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:39.677394 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 279/561] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 08:32:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 08:32:39.687007 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253d1321-79f0-43a0-89fe-9f21719beb3c None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:39.697087 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:39.733044 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6b1c61-6c73-4b17-960a-c80321dfb003 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:39.776729 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e82611b-b24d-417c-8571-ae7e222d810c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:39.779626 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e82611b-b24d-417c-8571-ae7e222d810c None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:39.780169 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e82611b-b24d-417c-8571-ae7e222d810c None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:39.780707 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e82611b-b24d-417c-8571-ae7e222d810c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:39.815359 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e82611b-b24d-417c-8571-ae7e222d810c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:32:39.820974 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-8e82611b-b24d-417c-8571-ae7e222d810c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 08:32:39.820974 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:39.820974 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:39.820974 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:39.820974 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:39.820974 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:39.820974 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:39.820974 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:39.820974 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:39.820974 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:39.820974 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:39.820974 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:39.820974 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:32:39.820974 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:32:39.820974 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:32:39.820974 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:32:39.822169 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:32:39.822169 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:32:39.822169 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:32:39.822169 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:32:39.822169 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:39.822169 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:39.822169 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:39.822169 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:39.822169 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:39.822169 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:39.822169 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:39.822169 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:39.822169 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:39.822169 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:39.822169 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:39.822169 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:39.823020 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:39.823020 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:39.823020 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:39.823020 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:39.823020 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:32:39.823020 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:32:39.823020 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:32:39.823020 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:32:39.823020 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:32:39.823020 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:32:39.823020 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 08:32:39.823020 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:39.823020 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 283/562] 10.222.0.30 () {56 vars in 1096 bytes} [Sat Oct 15 08:32:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:39.829768 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e82611b-b24d-417c-8571-ae7e222d810c None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:39.870833 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253d1321-79f0-43a0-89fe-9f21719beb3c None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:39.876065 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:39.911372 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95512b5c-5c0a-4bbd-80b2-317b7cb9b58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:39.912001 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95512b5c-5c0a-4bbd-80b2-317b7cb9b58e None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:39.912287 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95512b5c-5c0a-4bbd-80b2-317b7cb9b58e None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:39.912646 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95512b5c-5c0a-4bbd-80b2-317b7cb9b58e None admin] PATH_INFO: `/v3/projects/service` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:39.920345 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95512b5c-5c0a-4bbd-80b2-317b7cb9b58e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:39.921850 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95512b5c-5c0a-4bbd-80b2-317b7cb9b58e None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:39.935577 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-95512b5c-5c0a-4bbd-80b2-317b7cb9b58e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 08:32:39.935577 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:39.935577 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:39.935577 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:39.935577 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:39.935577 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:39.935577 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:39.935577 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:39.935577 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:39.935577 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:39.935577 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:39.935577 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:39.935577 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 08:32:39.935577 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 08:32:39.935577 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 08:32:39.936520 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 08:32:39.936520 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:39.936520 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:39.936520 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:39.936520 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:39.936520 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:39.936520 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:39.936520 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:39.936520 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:39.936520 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:39.936520 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:39.936520 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:39.936520 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:39.936520 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:39.936520 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:39.936520 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:39.936520 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:39.937403 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 08:32:39.937403 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 08:32:39.937403 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 08:32:39.937403 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 08:32:39.937403 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 08:32:39.937403 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 08:32:39.937403 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 08:32:39.937403 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:39.937403 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 280/563] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 08:32:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:39.942714 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95512b5c-5c0a-4bbd-80b2-317b7cb9b58e None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:39.953465 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e82611b-b24d-417c-8571-ae7e222d810c None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:40.002058 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce997006-9e92-4cef-8d99-d1afeb286c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:40.007347 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce997006-9e92-4cef-8d99-d1afeb286c43 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:40.007638 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce997006-9e92-4cef-8d99-d1afeb286c43 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:40.007923 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce997006-9e92-4cef-8d99-d1afeb286c43 None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:40.008666 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce997006-9e92-4cef-8d99-d1afeb286c43 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:40.008973 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce997006-9e92-4cef-8d99-d1afeb286c43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:40.010370 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce997006-9e92-4cef-8d99-d1afeb286c43 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:40.026075 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 284/564] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 08:32:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:40.031971 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce997006-9e92-4cef-8d99-d1afeb286c43 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:40.058609 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:40.111081 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95512b5c-5c0a-4bbd-80b2-317b7cb9b58e None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:40.161614 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:40.175547 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce997006-9e92-4cef-8d99-d1afeb286c43 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:40.180842 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf156ee0-3829-4526-b0a7-8b0cb7978ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:40.181463 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf156ee0-3829-4526-b0a7-8b0cb7978ab4 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:40.181767 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf156ee0-3829-4526-b0a7-8b0cb7978ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:40.182115 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf156ee0-3829-4526-b0a7-8b0cb7978ab4 None admin] PATH_INFO: `/v3/projects` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:40.182833 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf156ee0-3829-4526-b0a7-8b0cb7978ab4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:40.185702 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf156ee0-3829-4526-b0a7-8b0cb7978ab4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:40.185702 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf156ee0-3829-4526-b0a7-8b0cb7978ab4 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:40.197537 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 281/565] 10.222.0.30 () {56 vars in 1148 bytes} [Sat Oct 15 08:32:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 08:32:40.202836 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf156ee0-3829-4526-b0a7-8b0cb7978ab4 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:40.246947 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e3efcc-7993-4bb4-b99c-d269671dab6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:40.248896 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e3efcc-7993-4bb4-b99c-d269671dab6e None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:40.249290 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e3efcc-7993-4bb4-b99c-d269671dab6e None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:40.249690 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e3efcc-7993-4bb4-b99c-d269671dab6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:40.275742 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e3efcc-7993-4bb4-b99c-d269671dab6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:40.277408 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e3efcc-7993-4bb4-b99c-d269671dab6e None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:40.300027 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-08e3efcc-7993-4bb4-b99c-d269671dab6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:40.300027 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:40.300027 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:40.300027 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:40.300027 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:40.300027 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:40.300027 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:40.300027 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:40.300027 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:40.300027 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:40.300027 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:40.300027 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:40.300027 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 08:32:40.300027 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 08:32:40.300027 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 08:32:40.300027 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 08:32:40.301185 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:40.301185 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:40.301185 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:40.301185 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:40.301185 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:40.301185 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:40.301185 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:40.301185 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:40.301185 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:40.301185 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:40.301185 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:40.301185 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:40.301185 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:40.301185 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:40.301185 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:40.301185 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:40.301185 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 08:32:40.302090 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 08:32:40.302090 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:40.302090 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:40.302090 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 285/566] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:32:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:40.310654 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e3efcc-7993-4bb4-b99c-d269671dab6e None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:40.350047 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf156ee0-3829-4526-b0a7-8b0cb7978ab4 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:40.387563 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2d58ab-dee5-48ea-9230-5acb0a75e7cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:40.388317 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d58ab-dee5-48ea-9230-5acb0a75e7cb None admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:40.388706 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d58ab-dee5-48ea-9230-5acb0a75e7cb None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:40.389086 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d58ab-dee5-48ea-9230-5acb0a75e7cb None admin] PATH_INFO: `/v3/projects/5dfceb59b3794439a4586b3cbe408f0e/users/c5b2a0e623c348ad94d4820080677be6/roles/b1b0b29beb794324826a4c4003ae293b` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:40.402636 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:40.410022 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2d58ab-dee5-48ea-9230-5acb0a75e7cb None admin] RBAC: Authorizing `identity:create_grant(project_id=5dfceb59b3794439a4586b3cbe408f0e, user_id=c5b2a0e623c348ad94d4820080677be6, role_id=b1b0b29beb794324826a4c4003ae293b)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:40.411788 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2d58ab-dee5-48ea-9230-5acb0a75e7cb None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:40.428186 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 282/567] 10.222.0.30 () {58 vars in 1439 bytes} [Sat Oct 15 08:32:40 2022] PUT /identity/v3/projects/5dfceb59b3794439a4586b3cbe408f0e/users/c5b2a0e623c348ad94d4820080677be6/roles/b1b0b29beb794324826a4c4003ae293b => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:32:40.553318 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e3efcc-7993-4bb4-b99c-d269671dab6e None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:40.608076 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71abaa99-548f-4b29-81e1-87447e658f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:40.608713 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71abaa99-548f-4b29-81e1-87447e658f88 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:40.609001 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71abaa99-548f-4b29-81e1-87447e658f88 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:40.609282 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71abaa99-548f-4b29-81e1-87447e658f88 None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:40.609969 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71abaa99-548f-4b29-81e1-87447e658f88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:40.610280 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71abaa99-548f-4b29-81e1-87447e658f88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:40.611676 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71abaa99-548f-4b29-81e1-87447e658f88 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:40.636391 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 286/568] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 08:32:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:40.642395 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2d58ab-dee5-48ea-9230-5acb0a75e7cb None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:40.770582 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2d58ab-dee5-48ea-9230-5acb0a75e7cb None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:40.818744 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba23528a-8974-4310-8935-22fb51816f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:40.819760 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba23528a-8974-4310-8935-22fb51816f04 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:40.820264 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba23528a-8974-4310-8935-22fb51816f04 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:40.821326 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba23528a-8974-4310-8935-22fb51816f04 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:40.829304 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba23528a-8974-4310-8935-22fb51816f04 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:40.831060 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba23528a-8974-4310-8935-22fb51816f04 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:40.847879 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-ba23528a-8974-4310-8935-22fb51816f04 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 15 08:32:40.847879 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:40.847879 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:40.847879 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:40.847879 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:40.847879 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:40.847879 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:40.847879 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:40.847879 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:40.847879 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:40.847879 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:40.847879 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:40.847879 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 08:32:40.847879 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 08:32:40.847879 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 08:32:40.847879 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 08:32:40.848837 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:40.848837 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:40.848837 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 08:32:40.848837 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 08:32:40.848837 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 08:32:40.848837 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 08:32:40.848837 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:40.848837 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:40.848837 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:40.848837 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:40.848837 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:40.848837 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:40.848837 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:40.848837 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:40.848837 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:40.848837 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:40.848837 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:40.848837 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:40.849816 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:40.849816 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:40.849816 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 08:32:40.849816 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 08:32:40.849816 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 08:32:40.849816 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 08:32:40.849816 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 08:32:40.849816 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 08:32:40.849816 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 15 08:32:40.849816 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:40.852556 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 283/569] 10.222.0.30 () {56 vars in 1096 bytes} [Sat Oct 15 08:32:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:40.858579 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71abaa99-548f-4b29-81e1-87447e658f88 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:40.883775 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:40.998128 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71abaa99-548f-4b29-81e1-87447e658f88 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:41.046972 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98c4354-586b-4bac-9af5-62b2b7bd3c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:41.047598 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98c4354-586b-4bac-9af5-62b2b7bd3c77 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:41.047886 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98c4354-586b-4bac-9af5-62b2b7bd3c77 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:41.048155 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98c4354-586b-4bac-9af5-62b2b7bd3c77 None admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:41.048900 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98c4354-586b-4bac-9af5-62b2b7bd3c77 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:41.049210 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98c4354-586b-4bac-9af5-62b2b7bd3c77 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:41.050669 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98c4354-586b-4bac-9af5-62b2b7bd3c77 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:41.068950 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 287/570] 10.222.0.30 () {56 vars in 1135 bytes} [Sat Oct 15 08:32:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:41.078755 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba23528a-8974-4310-8935-22fb51816f04 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:41.132661 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:41.220022 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba23528a-8974-4310-8935-22fb51816f04 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:41.254539 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7751700-71d6-43f8-ad9b-234bf8624167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:41.255104 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7751700-71d6-43f8-ad9b-234bf8624167 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:41.255385 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7751700-71d6-43f8-ad9b-234bf8624167 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:41.255656 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7751700-71d6-43f8-ad9b-234bf8624167 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:41.267413 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7751700-71d6-43f8-ad9b-234bf8624167 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:41.268864 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7751700-71d6-43f8-ad9b-234bf8624167 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:41.283422 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-b7751700-71d6-43f8-ad9b-234bf8624167 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:41.283422 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:41.283422 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:41.283422 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:41.283422 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:41.283422 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:41.283422 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:41.283422 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:41.283422 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:41.283422 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:41.283422 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:41.283422 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:41.283422 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 08:32:41.283422 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 08:32:41.283422 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 08:32:41.283422 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 08:32:41.284491 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:41.284491 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:41.284491 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:41.284491 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:41.284491 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:41.284491 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:41.284491 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:41.284491 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:41.284491 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:41.284491 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:41.284491 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:41.284491 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:41.284491 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:41.284491 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:41.284491 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:41.284491 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:41.284491 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 08:32:41.285412 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 08:32:41.285412 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:41.285412 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:41.285412 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 284/571] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:32:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:41.290349 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98c4354-586b-4bac-9af5-62b2b7bd3c77 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:41.398817 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:41.442993 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98c4354-586b-4bac-9af5-62b2b7bd3c77 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:41.490395 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6300b3a0-bea1-4838-9ec5-242f91236696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:41.491008 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6300b3a0-bea1-4838-9ec5-242f91236696 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:41.491294 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6300b3a0-bea1-4838-9ec5-242f91236696 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:41.491571 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6300b3a0-bea1-4838-9ec5-242f91236696 None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:41.492226 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6300b3a0-bea1-4838-9ec5-242f91236696 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:41.493136 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6300b3a0-bea1-4838-9ec5-242f91236696 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:41.494558 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6300b3a0-bea1-4838-9ec5-242f91236696 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:41.512451 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 288/572] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 08:32:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:41.518284 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7751700-71d6-43f8-ad9b-234bf8624167 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:41.552318 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c14244d-346e-4252-bffc-e31c46466dd9 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:41.552625 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c14244d-346e-4252-bffc-e31c46466dd9 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:41.552892 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c14244d-346e-4252-bffc-e31c46466dd9 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:41.554416 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 289/573] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:32:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 08:32:41.560413 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacbb255-07fc-41f3-b98a-4495402e4057 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:41.560704 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacbb255-07fc-41f3-b98a-4495402e4057 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:41.561461 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacbb255-07fc-41f3-b98a-4495402e4057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:41.581283 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-bacbb255-07fc-41f3-b98a-4495402e4057 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:32:41.583289 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-bacbb255-07fc-41f3-b98a-4495402e4057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:32:41.677734 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:41.689658 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 290/574] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:32:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 08:32:41.696825 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7751700-71d6-43f8-ad9b-234bf8624167 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:41.731252 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b3e17-3767-4dcd-bf8c-6cb344bea239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:41.731886 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b3e17-3767-4dcd-bf8c-6cb344bea239 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:41.732169 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b3e17-3767-4dcd-bf8c-6cb344bea239 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:41.732445 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b3e17-3767-4dcd-bf8c-6cb344bea239 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:41.739154 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2b3e17-3767-4dcd-bf8c-6cb344bea239 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:41.740604 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2b3e17-3767-4dcd-bf8c-6cb344bea239 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:41.752003 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-1b2b3e17-3767-4dcd-bf8c-6cb344bea239 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 08:32:41.752003 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:41.752003 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:41.752003 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:41.752003 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:41.752003 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:41.752003 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:41.752003 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:41.752003 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:41.752003 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:41.752003 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:41.752003 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:41.752003 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 08:32:41.752003 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 08:32:41.752003 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 08:32:41.752909 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 08:32:41.752909 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:41.752909 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:41.752909 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:41.752909 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:41.752909 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:41.752909 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:41.752909 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:41.752909 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:41.752909 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:41.752909 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:41.752909 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:41.752909 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:41.752909 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:41.752909 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:41.752909 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:41.752909 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:41.754214 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 08:32:41.754214 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 08:32:41.754214 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 08:32:41.754214 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 08:32:41.754214 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 08:32:41.754214 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 08:32:41.754214 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 08:32:41.754214 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:41.754214 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 285/575] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 08:32:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:41.762353 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacbb255-07fc-41f3-b98a-4495402e4057 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:41.838800 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:41.894537 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacbb255-07fc-41f3-b98a-4495402e4057 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:41.940598 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fccff1b-37e0-4438-98dc-bc8371384ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:41.941209 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fccff1b-37e0-4438-98dc-bc8371384ae6 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:41.941495 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fccff1b-37e0-4438-98dc-bc8371384ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:41.941796 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fccff1b-37e0-4438-98dc-bc8371384ae6 None admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:41.942492 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fccff1b-37e0-4438-98dc-bc8371384ae6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:41.942803 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fccff1b-37e0-4438-98dc-bc8371384ae6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:41.944232 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fccff1b-37e0-4438-98dc-bc8371384ae6 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:41.956839 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 291/576] 10.222.0.30 () {56 vars in 1148 bytes} [Sat Oct 15 08:32:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:41.962811 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b3e17-3767-4dcd-bf8c-6cb344bea239 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:42.000142 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fccff1b-37e0-4438-98dc-bc8371384ae6 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:42.135140 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b3e17-3767-4dcd-bf8c-6cb344bea239 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:42.137200 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fccff1b-37e0-4438-98dc-bc8371384ae6 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:42.162390 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:42.170684 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee6bd92-0c0d-4b51-96bc-8559abf61ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:42.171348 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee6bd92-0c0d-4b51-96bc-8559abf61ca1 None admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:42.171629 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee6bd92-0c0d-4b51-96bc-8559abf61ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:42.171903 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee6bd92-0c0d-4b51-96bc-8559abf61ca1 None admin] PATH_INFO: `/v3/projects/5dfceb59b3794439a4586b3cbe408f0e/users/6e3009e20e4c49e5a74f7fd793b2c329/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:42.173848 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee6bd92-0c0d-4b51-96bc-8559abf61ca1 None admin] RBAC: Authorizing `identity:create_grant(project_id=5dfceb59b3794439a4586b3cbe408f0e, user_id=6e3009e20e4c49e5a74f7fd793b2c329, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:42.175634 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee6bd92-0c0d-4b51-96bc-8559abf61ca1 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:42.177887 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b5c8a0-ce2e-4b1d-ba77-06c6c245cabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:42.178840 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5c8a0-ce2e-4b1d-ba77-06c6c245cabd None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:42.179662 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5c8a0-ce2e-4b1d-ba77-06c6c245cabd None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:42.180635 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5c8a0-ce2e-4b1d-ba77-06c6c245cabd None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:42.181267 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b5c8a0-ce2e-4b1d-ba77-06c6c245cabd None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:42.182759 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b5c8a0-ce2e-4b1d-ba77-06c6c245cabd None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:42.193228 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-43b5c8a0-ce2e-4b1d-ba77-06c6c245cabd None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 15 08:32:42.193228 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:42.193228 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 08:32:42.193228 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 08:32:42.193228 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 08:32:42.193228 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 08:32:42.193228 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 08:32:42.193228 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 08:32:42.193228 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 15 08:32:42.193228 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:42.193228 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 08:32:42.193228 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:42.193228 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:42.193228 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:42.193228 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:42.193228 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:42.193228 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:42.194833 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:42.194833 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:42.194833 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:42.194833 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:42.194833 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:42.194833 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:42.194833 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 08:32:42.194833 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 08:32:42.194833 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 08:32:42.194833 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 08:32:42.194833 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:42.194833 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:42.194833 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:42.194833 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:42.194833 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:42.194833 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:42.196061 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:42.196061 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:42.196061 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:42.196061 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:42.196061 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:42.196061 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:42.196061 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:42.196061 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:42.196061 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:42.196061 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:42.196061 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 08:32:42.196061 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 08:32:42.196061 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 15 08:32:42.196061 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:42.197121 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 286/577] 10.222.0.30 () {58 vars in 1439 bytes} [Sat Oct 15 08:32:41 2022] PUT /identity/v3/projects/5dfceb59b3794439a4586b3cbe408f0e/users/6e3009e20e4c49e5a74f7fd793b2c329/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:32:42.197208 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 292/578] 10.222.0.30 () {56 vars in 1129 bytes} [Sat Oct 15 08:32:41 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 08:32:42.199940 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f0409-a417-4f60-8301-45afb9559403 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:42.199940 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f0409-a417-4f60-8301-45afb9559403 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:42.199940 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f0409-a417-4f60-8301-45afb9559403 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:42.202109 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b5c8a0-ce2e-4b1d-ba77-06c6c245cabd None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:42.202257 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 287/579] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:32:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:32:42.209904 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9f00d-2aff-4c0b-b729-73a4e24ed432 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:42.210188 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9f00d-2aff-4c0b-b729-73a4e24ed432 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:42.210467 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9f00d-2aff-4c0b-b729-73a4e24ed432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:42.247290 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-b7a9f00d-2aff-4c0b-b729-73a4e24ed432 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:32:42.248534 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-b7a9f00d-2aff-4c0b-b729-73a4e24ed432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:32:42.251815 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:42.363150 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b5c8a0-ce2e-4b1d-ba77-06c6c245cabd None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:42.365368 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 288/580] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:32:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 08:32:42.366839 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c410158-d079-456c-b7ee-adbda05f9bcb None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:42.367126 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c410158-d079-456c-b7ee-adbda05f9bcb None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:42.367445 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c410158-d079-456c-b7ee-adbda05f9bcb None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:42.372955 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 289/581] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:32:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:32:42.374238 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9eb3c2-9e5c-4432-9fa0-9be59c5398f2 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:42.374625 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9eb3c2-9e5c-4432-9fa0-9be59c5398f2 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:42.374910 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9eb3c2-9e5c-4432-9fa0-9be59c5398f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:42.401798 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee47bb0f-0b0d-483b-8314-69c2c0b8cb20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:42.401798 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee47bb0f-0b0d-483b-8314-69c2c0b8cb20 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:42.402056 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee47bb0f-0b0d-483b-8314-69c2c0b8cb20 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:42.402274 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee47bb0f-0b0d-483b-8314-69c2c0b8cb20 None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:42.402803 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee47bb0f-0b0d-483b-8314-69c2c0b8cb20 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:42.403109 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee47bb0f-0b0d-483b-8314-69c2c0b8cb20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:42.408231 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee47bb0f-0b0d-483b-8314-69c2c0b8cb20 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:42.409949 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 293/582] 10.222.0.30 () {56 vars in 1124 bytes} [Sat Oct 15 08:32:42 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:42.415968 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-fa9eb3c2-9e5c-4432-9fa0-9be59c5398f2 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:32:42.417197 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-fa9eb3c2-9e5c-4432-9fa0-9be59c5398f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:32:42.420220 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee47bb0f-0b0d-483b-8314-69c2c0b8cb20 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:42.479522 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:42.515103 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 290/583] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:32:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 08:32:42.548200 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee47bb0f-0b0d-483b-8314-69c2c0b8cb20 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:42.582609 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b045a907-d388-4728-b2b5-5a8215f49b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:42.583221 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b045a907-d388-4728-b2b5-5a8215f49b5d None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:42.583508 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b045a907-d388-4728-b2b5-5a8215f49b5d None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:42.583788 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b045a907-d388-4728-b2b5-5a8215f49b5d None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:42.584416 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b045a907-d388-4728-b2b5-5a8215f49b5d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:42.584726 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b045a907-d388-4728-b2b5-5a8215f49b5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:42.586220 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b045a907-d388-4728-b2b5-5a8215f49b5d None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:42.591753 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 294/584] 10.222.0.30 () {56 vars in 1124 bytes} [Sat Oct 15 08:32:42 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 08:32:42.597181 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9eb3c2-9e5c-4432-9fa0-9be59c5398f2 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:42.692687 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b045a907-d388-4728-b2b5-5a8215f49b5d None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:42.711098 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:42.768214 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9eb3c2-9e5c-4432-9fa0-9be59c5398f2 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:42.842644 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0211d8b-da68-4ba1-8bf9-a73097202a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:42.843262 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0211d8b-da68-4ba1-8bf9-a73097202a29 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:42.843538 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0211d8b-da68-4ba1-8bf9-a73097202a29 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:42.847623 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0211d8b-da68-4ba1-8bf9-a73097202a29 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:42.848346 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0211d8b-da68-4ba1-8bf9-a73097202a29 None admin] RBAC: Adding query filter params (interface=public, service_id=f82e1ab33ab44041a4ae37dea257e5ca, region_id=RegionOne) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:42.848738 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0211d8b-da68-4ba1-8bf9-a73097202a29 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:42.850155 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0211d8b-da68-4ba1-8bf9-a73097202a29 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:42.865414 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 291/585] 10.222.0.30 () {56 vars in 1251 bytes} [Sat Oct 15 08:32:42 2022] GET /identity/v3/endpoints?service_id=f82e1ab33ab44041a4ae37dea257e5ca&interface=public®ion_id=RegionOne => generated 194 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:42.866986 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0211d8b-da68-4ba1-8bf9-a73097202a29 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:42.876101 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b045a907-d388-4728-b2b5-5a8215f49b5d None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:42.886413 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:42.913968 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d6ac40-5c1d-4198-ac16-7ac995c898d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:42.914755 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d6ac40-5c1d-4198-ac16-7ac995c898d5 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:42.915100 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d6ac40-5c1d-4198-ac16-7ac995c898d5 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:42.915489 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d6ac40-5c1d-4198-ac16-7ac995c898d5 None admin] PATH_INFO: `/v3/services/network` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:42.916176 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d6ac40-5c1d-4198-ac16-7ac995c898d5 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:42.917595 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d6ac40-5c1d-4198-ac16-7ac995c898d5 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:42.926702 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-89d6ac40-5c1d-4198-ac16-7ac995c898d5 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 15 08:32:42.926702 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:42.926702 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 08:32:42.926702 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 08:32:42.926702 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 08:32:42.926702 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 08:32:42.926702 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 08:32:42.926702 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 08:32:42.926702 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 15 08:32:42.926702 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:42.926702 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 08:32:42.926702 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:42.926702 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:42.926702 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:42.926702 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:42.926702 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:42.926702 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:42.927824 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:42.927824 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:42.927824 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:42.927824 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:42.927824 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:42.927824 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:42.927824 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 08:32:42.927824 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 08:32:42.927824 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 08:32:42.927824 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 08:32:42.927824 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:42.927824 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:42.927824 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:42.927824 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:42.927824 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:42.927824 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:42.928811 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:42.928811 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:42.928811 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:42.928811 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:42.928811 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:42.928811 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:42.928811 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:42.928811 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:42.928811 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:42.928811 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:42.928811 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 08:32:42.928811 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 08:32:42.928811 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 15 08:32:42.928811 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:42.928811 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 295/586] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 08:32:42 2022] GET /identity/v3/services/network => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 08:32:42.929649 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d6ac40-5c1d-4198-ac16-7ac995c898d5 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:43.005186 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0211d8b-da68-4ba1-8bf9-a73097202a29 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:43.037160 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d6ac40-5c1d-4198-ac16-7ac995c898d5 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:43.050191 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:43.056022 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbf7ed5-d3ee-464c-8324-6f8cde901a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:43.057093 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbf7ed5-d3ee-464c-8324-6f8cde901a4e None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:43.057576 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbf7ed5-d3ee-464c-8324-6f8cde901a4e None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:43.058172 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbf7ed5-d3ee-464c-8324-6f8cde901a4e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:43.059103 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbf7ed5-d3ee-464c-8324-6f8cde901a4e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:43.060827 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbf7ed5-d3ee-464c-8324-6f8cde901a4e None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:43.070543 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-dbbf7ed5-d3ee-464c-8324-6f8cde901a4e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 15 08:32:43.070543 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:43.070543 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 08:32:43.070543 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 08:32:43.070543 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 08:32:43.070543 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 08:32:43.070543 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 08:32:43.070543 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 08:32:43.070543 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 15 08:32:43.070543 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:43.070543 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 08:32:43.070543 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:43.070543 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:43.070543 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:43.070543 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:43.070543 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:43.071727 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:43.071727 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:43.071727 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:43.071727 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:43.071727 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:43.071727 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:43.071727 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:43.071727 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 08:32:43.071727 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 08:32:43.071727 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 08:32:43.071727 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 08:32:43.071727 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:43.071727 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:43.071727 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:43.071727 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:43.071727 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:43.071727 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:43.072842 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:43.072842 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:43.072842 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:43.072842 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:43.072842 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:43.072842 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:43.072842 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:43.072842 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:43.072842 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:43.072842 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:43.072842 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 08:32:43.072842 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 08:32:43.072842 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 15 08:32:43.072842 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:43.079649 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 292/587] 10.222.0.30 () {56 vars in 1132 bytes} [Sat Oct 15 08:32:42 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:43.079649 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbf7ed5-d3ee-464c-8324-6f8cde901a4e None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:43.096875 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8470efa1-d7ea-41e9-bd3a-295c2100de0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:43.098156 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8470efa1-d7ea-41e9-bd3a-295c2100de0f None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:43.098606 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8470efa1-d7ea-41e9-bd3a-295c2100de0f None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:43.099035 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8470efa1-d7ea-41e9-bd3a-295c2100de0f None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:43.099954 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8470efa1-d7ea-41e9-bd3a-295c2100de0f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:43.102580 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8470efa1-d7ea-41e9-bd3a-295c2100de0f None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:43.116442 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 296/588] 10.222.0.30 () {56 vars in 1087 bytes} [Sat Oct 15 08:32:42 2022] GET /identity/v3/services => generated 772 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:43.116442 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8470efa1-d7ea-41e9-bd3a-295c2100de0f None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:43.215364 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:43.276181 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbf7ed5-d3ee-464c-8324-6f8cde901a4e None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:43.301798 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8470efa1-d7ea-41e9-bd3a-295c2100de0f None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:43.355226 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a48e228-d065-4012-90c1-538ce67c8d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:43.355851 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a48e228-d065-4012-90c1-538ce67c8d88 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:43.356131 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a48e228-d065-4012-90c1-538ce67c8d88 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:43.356404 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a48e228-d065-4012-90c1-538ce67c8d88 None admin] PATH_INFO: `/v3/services` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:43.357104 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a48e228-d065-4012-90c1-538ce67c8d88 None admin] RBAC: Adding query filter params (name=network) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:43.357413 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a48e228-d065-4012-90c1-538ce67c8d88 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:43.358875 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a48e228-d065-4012-90c1-538ce67c8d88 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:43.360525 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ee576d-0899-4ebc-915d-b12527671b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:43.361165 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee576d-0899-4ebc-915d-b12527671b09 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:43.361532 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee576d-0899-4ebc-915d-b12527671b09 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:43.361846 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee576d-0899-4ebc-915d-b12527671b09 None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:43.362479 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ee576d-0899-4ebc-915d-b12527671b09 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:43.362785 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ee576d-0899-4ebc-915d-b12527671b09 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:43.364133 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ee576d-0899-4ebc-915d-b12527671b09 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:43.365456 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 293/589] 10.222.0.30 () {56 vars in 1112 bytes} [Sat Oct 15 08:32:43 2022] GET /identity/v3/services?name=network => generated 124 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:43.366889 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf9a14e-b28d-427f-afec-3eb3765ffc0e None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:43.369206 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf9a14e-b28d-427f-afec-3eb3765ffc0e None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:43.369741 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf9a14e-b28d-427f-afec-3eb3765ffc0e None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:43.371523 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 294/591] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:32:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:32:43.371743 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 297/591] 10.222.0.30 () {56 vars in 1126 bytes} [Sat Oct 15 08:32:43 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:43.372658 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf9a14e-b28d-427f-afec-3eb3765ffc0e None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:43.375262 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ee576d-0899-4ebc-915d-b12527671b09 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:43.455177 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:43.491099 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ee576d-0899-4ebc-915d-b12527671b09 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:43.494370 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:43.504076 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf9a14e-b28d-427f-afec-3eb3765ffc0e None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:43.534899 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54173b24-3ea5-4096-8f1d-2174a0289554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:43.535437 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54173b24-3ea5-4096-8f1d-2174a0289554 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:43.535722 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54173b24-3ea5-4096-8f1d-2174a0289554 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:43.536001 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54173b24-3ea5-4096-8f1d-2174a0289554 None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:43.536630 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54173b24-3ea5-4096-8f1d-2174a0289554 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:43.536939 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54173b24-3ea5-4096-8f1d-2174a0289554 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:43.538446 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54173b24-3ea5-4096-8f1d-2174a0289554 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:43.543796 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 298/592] 10.222.0.30 () {56 vars in 1126 bytes} [Sat Oct 15 08:32:43 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 08:32:43.544685 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00cadd6-eaa2-47c8-8687-de400301d33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:43.545097 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e537b7f1-701b-4064-a25d-4494de46b92e None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:43.545384 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e537b7f1-701b-4064-a25d-4494de46b92e None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:43.545660 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e537b7f1-701b-4064-a25d-4494de46b92e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:43.546232 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00cadd6-eaa2-47c8-8687-de400301d33c None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:43.546717 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00cadd6-eaa2-47c8-8687-de400301d33c None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:43.547204 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00cadd6-eaa2-47c8-8687-de400301d33c None admin] PATH_INFO: `/v3/services` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:43.548119 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00cadd6-eaa2-47c8-8687-de400301d33c None admin] RBAC: Adding query filter params (type=network) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:43.548797 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00cadd6-eaa2-47c8-8687-de400301d33c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:43.550648 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00cadd6-eaa2-47c8-8687-de400301d33c None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:43.557305 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 295/593] 10.222.0.30 () {56 vars in 1112 bytes} [Sat Oct 15 08:32:43 2022] GET /identity/v3/services?type=network => generated 350 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:43.567587 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00cadd6-eaa2-47c8-8687-de400301d33c None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:43.572402 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-e537b7f1-701b-4064-a25d-4494de46b92e None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:32:43.573636 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-e537b7f1-701b-4064-a25d-4494de46b92e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:32:43.722828 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 299/594] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:32:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 08:32:43.740875 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00cadd6-eaa2-47c8-8687-de400301d33c None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:43.781270 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:43.792626 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0e78ae-4d3b-4afa-a0e2-76b14946aa75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:43.793294 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0e78ae-4d3b-4afa-a0e2-76b14946aa75 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:43.793647 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0e78ae-4d3b-4afa-a0e2-76b14946aa75 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:43.794023 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0e78ae-4d3b-4afa-a0e2-76b14946aa75 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:43.794858 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0e78ae-4d3b-4afa-a0e2-76b14946aa75 None admin] RBAC: Adding query filter params (interface=public, service_id=919cf4ab5e73419f86f50ea2af2fb754, region_id=RegionOne) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:43.795239 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0e78ae-4d3b-4afa-a0e2-76b14946aa75 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:43.796685 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0e78ae-4d3b-4afa-a0e2-76b14946aa75 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:43.802376 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 296/595] 10.222.0.30 () {56 vars in 1251 bytes} [Sat Oct 15 08:32:43 2022] GET /identity/v3/endpoints?service_id=919cf4ab5e73419f86f50ea2af2fb754&interface=public®ion_id=RegionOne => generated 194 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:43.808645 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e537b7f1-701b-4064-a25d-4494de46b92e None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:43.911730 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:43.919852 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e537b7f1-701b-4064-a25d-4494de46b92e None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:43.954617 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d684ea6-faaf-41e8-9c81-c92b49cde2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:43.955233 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d684ea6-faaf-41e8-9c81-c92b49cde2bd None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:43.955517 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d684ea6-faaf-41e8-9c81-c92b49cde2bd None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:43.955795 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d684ea6-faaf-41e8-9c81-c92b49cde2bd None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:43.956398 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d684ea6-faaf-41e8-9c81-c92b49cde2bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:43.957840 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d684ea6-faaf-41e8-9c81-c92b49cde2bd None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:43.963336 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 300/596] 10.222.0.30 () {56 vars in 1087 bytes} [Sat Oct 15 08:32:43 2022] GET /identity/v3/services => generated 772 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:44.095987 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0e78ae-4d3b-4afa-a0e2-76b14946aa75 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:44.230973 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0e78ae-4d3b-4afa-a0e2-76b14946aa75 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:44.277619 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bce96f-4a46-4390-97f4-6276be5a7589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:44.278287 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bce96f-4a46-4390-97f4-6276be5a7589 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:44.278568 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bce96f-4a46-4390-97f4-6276be5a7589 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:44.278844 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bce96f-4a46-4390-97f4-6276be5a7589 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:44.289715 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bce96f-4a46-4390-97f4-6276be5a7589 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:32:44.295001 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-e2bce96f-4a46-4390-97f4-6276be5a7589 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 15 08:32:44.295001 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:44.295001 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:44.295001 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:44.295001 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:44.295001 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:44.295001 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:44.295001 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:44.295001 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:44.295001 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:44.295001 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:44.295001 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:44.295001 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:32:44.295001 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:32:44.295001 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:32:44.295001 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:32:44.295940 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:32:44.295940 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:32:44.295940 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:32:44.295940 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:32:44.295940 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:44.295940 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:44.295940 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:44.295940 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:44.295940 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:44.295940 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:44.295940 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:44.295940 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:44.295940 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:44.295940 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:44.295940 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:44.295940 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:44.296777 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:44.296777 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:44.296777 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:44.296777 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:44.296777 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:32:44.296777 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:32:44.296777 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:32:44.296777 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:32:44.296777 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:32:44.296777 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:32:44.296777 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 15 08:32:44.296777 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:44.296777 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 297/597] 10.222.0.30 () {56 vars in 1102 bytes} [Sat Oct 15 08:32:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:44.302272 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d684ea6-faaf-41e8-9c81-c92b49cde2bd None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:44.417087 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d684ea6-faaf-41e8-9c81-c92b49cde2bd None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:44.451541 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97243493-434c-4b6d-b565-308a9cda6e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:44.452197 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97243493-434c-4b6d-b565-308a9cda6e1d None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:44.452482 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97243493-434c-4b6d-b565-308a9cda6e1d None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:44.452849 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97243493-434c-4b6d-b565-308a9cda6e1d None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:44.453522 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97243493-434c-4b6d-b565-308a9cda6e1d None admin] RBAC: Adding query filter params (name=service) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:44.453852 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97243493-434c-4b6d-b565-308a9cda6e1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:44.455212 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97243493-434c-4b6d-b565-308a9cda6e1d None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:44.464240 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 301/598] 10.222.0.30 () {56 vars in 1103 bytes} [Sat Oct 15 08:32:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:44.469518 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bce96f-4a46-4390-97f4-6276be5a7589 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:44.537956 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:44.627976 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bce96f-4a46-4390-97f4-6276be5a7589 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:44.684190 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c732b7ed-f87a-44f7-b8c4-356de53ebc57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:44.684768 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c732b7ed-f87a-44f7-b8c4-356de53ebc57 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:44.685052 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c732b7ed-f87a-44f7-b8c4-356de53ebc57 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:44.685325 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c732b7ed-f87a-44f7-b8c4-356de53ebc57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:44.706282 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c732b7ed-f87a-44f7-b8c4-356de53ebc57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:44.707795 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c732b7ed-f87a-44f7-b8c4-356de53ebc57 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:44.728438 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-c732b7ed-f87a-44f7-b8c4-356de53ebc57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:44.728438 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:44.728438 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:44.728438 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:44.728438 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:44.728438 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:44.728438 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:44.728438 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:44.728438 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:44.728438 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:44.728438 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:44.728438 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:44.728438 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 08:32:44.728438 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 08:32:44.728438 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 08:32:44.728438 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 08:32:44.729493 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:44.729493 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:44.729493 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:44.729493 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:44.729493 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:44.729493 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:44.729493 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:44.729493 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:44.729493 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:44.729493 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:44.729493 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:44.729493 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:44.729493 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:44.729493 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:44.729493 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:44.729493 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:44.729493 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 08:32:44.730443 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 08:32:44.730443 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:44.730443 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:44.734102 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 298/599] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:32:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:44.735380 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97243493-434c-4b6d-b565-308a9cda6e1d None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:44.769486 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:44.785439 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad2dbe1-0749-4bd9-b9cc-8b9323bfd7fd None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:44.786500 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad2dbe1-0749-4bd9-b9cc-8b9323bfd7fd None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:44.787121 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad2dbe1-0749-4bd9-b9cc-8b9323bfd7fd None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:44.788958 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 299/600] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:32:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:32:44.794327 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc972266-1621-4cd6-ae22-7947b776137f None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:44.794907 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc972266-1621-4cd6-ae22-7947b776137f None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:44.795386 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc972266-1621-4cd6-ae22-7947b776137f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:44.817564 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-fc972266-1621-4cd6-ae22-7947b776137f None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:32:44.818798 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-fc972266-1621-4cd6-ae22-7947b776137f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:32:44.855969 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97243493-434c-4b6d-b565-308a9cda6e1d None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:44.899087 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0c2233-9fd9-4f1f-839c-af0c51849b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:44.899702 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c2233-9fd9-4f1f-839c-af0c51849b13 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:44.899986 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c2233-9fd9-4f1f-839c-af0c51849b13 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:44.900263 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c2233-9fd9-4f1f-839c-af0c51849b13 None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:44.900909 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0c2233-9fd9-4f1f-839c-af0c51849b13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:44.901214 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0c2233-9fd9-4f1f-839c-af0c51849b13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:44.902629 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0c2233-9fd9-4f1f-839c-af0c51849b13 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:44.925802 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 300/601] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:32:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 08:32:44.929861 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 302/602] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 08:32:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:44.935392 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc972266-1621-4cd6-ae22-7947b776137f None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:44.976443 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:45.069505 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc972266-1621-4cd6-ae22-7947b776137f None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:45.109360 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b41383-f2b0-460c-80ec-c6384bf013dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:45.110031 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b41383-f2b0-460c-80ec-c6384bf013dd None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:45.110312 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b41383-f2b0-460c-80ec-c6384bf013dd None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:45.110582 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b41383-f2b0-460c-80ec-c6384bf013dd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:45.121594 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b41383-f2b0-460c-80ec-c6384bf013dd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:45.123077 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b41383-f2b0-460c-80ec-c6384bf013dd None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:45.141102 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-d1b41383-f2b0-460c-80ec-c6384bf013dd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 15 08:32:45.141102 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:45.141102 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:45.141102 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:45.141102 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:45.141102 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:45.141102 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:45.141102 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:45.141102 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:45.141102 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:45.141102 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:45.141102 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:45.141102 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 08:32:45.141102 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 08:32:45.141102 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 08:32:45.141102 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 08:32:45.142414 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:45.142414 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:45.142414 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 08:32:45.142414 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 08:32:45.142414 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 08:32:45.142414 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 08:32:45.142414 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:45.142414 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:45.142414 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:45.142414 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:45.142414 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:45.142414 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:45.142414 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:45.142414 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:45.142414 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:45.142414 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:45.142414 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:45.142414 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:45.144189 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:45.144189 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:45.144189 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 08:32:45.144189 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 08:32:45.144189 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 08:32:45.144189 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 08:32:45.144189 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 08:32:45.144189 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 08:32:45.144189 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 15 08:32:45.144189 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:45.144189 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 301/603] 10.222.0.30 () {56 vars in 1117 bytes} [Sat Oct 15 08:32:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:45.150783 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0c2233-9fd9-4f1f-839c-af0c51849b13 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:45.265658 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:45.289384 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0c2233-9fd9-4f1f-839c-af0c51849b13 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:45.331485 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b9e582-1122-45c6-a078-20974cc0d287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:45.332470 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b9e582-1122-45c6-a078-20974cc0d287 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:45.332957 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b9e582-1122-45c6-a078-20974cc0d287 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:45.333451 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b9e582-1122-45c6-a078-20974cc0d287 None admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:45.334414 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b9e582-1122-45c6-a078-20974cc0d287 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:45.334948 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b9e582-1122-45c6-a078-20974cc0d287 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:45.336639 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b9e582-1122-45c6-a078-20974cc0d287 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:45.359646 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 303/604] 10.222.0.30 () {56 vars in 1149 bytes} [Sat Oct 15 08:32:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:45.364842 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b41383-f2b0-460c-80ec-c6384bf013dd None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:45.399592 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b9e582-1122-45c6-a078-20974cc0d287 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:45.539986 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:45.554707 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b41383-f2b0-460c-80ec-c6384bf013dd None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:45.559669 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b9e582-1122-45c6-a078-20974cc0d287 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:45.614630 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002a976c-3973-4273-b0bd-356c1a210765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:45.615663 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002a976c-3973-4273-b0bd-356c1a210765 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:45.616161 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002a976c-3973-4273-b0bd-356c1a210765 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:45.616666 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002a976c-3973-4273-b0bd-356c1a210765 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:45.627777 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:45.634295 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002a976c-3973-4273-b0bd-356c1a210765 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:32:45.639376 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-002a976c-3973-4273-b0bd-356c1a210765 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 08:32:45.639376 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:45.639376 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:45.639376 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:45.639376 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:45.639376 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:45.639376 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:45.639376 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:45.639376 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:45.639376 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:45.639376 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:45.639376 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:45.639376 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:32:45.639376 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:32:45.639376 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:32:45.639376 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:32:45.640328 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:32:45.640328 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:32:45.640328 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:32:45.640328 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:32:45.640328 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:45.640328 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:45.640328 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:45.640328 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:45.640328 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:45.640328 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:45.640328 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:45.640328 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:45.640328 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:45.640328 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:45.640328 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:45.640328 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:45.641171 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:45.641171 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:45.641171 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:45.641171 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:45.641171 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:32:45.641171 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:32:45.641171 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:32:45.641171 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:32:45.641171 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:32:45.641171 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:32:45.641171 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 08:32:45.641171 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:45.645662 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 304/605] 10.222.0.30 () {56 vars in 1096 bytes} [Sat Oct 15 08:32:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:45.647407 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9305075c-244f-47ca-8008-e5e0c6623007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:45.648261 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9305075c-244f-47ca-8008-e5e0c6623007 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:45.648803 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9305075c-244f-47ca-8008-e5e0c6623007 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:45.649303 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9305075c-244f-47ca-8008-e5e0c6623007 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:45.653159 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002a976c-3973-4273-b0bd-356c1a210765 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:45.784143 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002a976c-3973-4273-b0bd-356c1a210765 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:45.810095 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9305075c-244f-47ca-8008-e5e0c6623007 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:45.811344 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9305075c-244f-47ca-8008-e5e0c6623007 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:45.831990 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a7593a-ccbb-410b-b565-88525561038f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:45.834061 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a7593a-ccbb-410b-b565-88525561038f None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:45.835859 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a7593a-ccbb-410b-b565-88525561038f None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:45.836916 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a7593a-ccbb-410b-b565-88525561038f None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:45.839396 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a7593a-ccbb-410b-b565-88525561038f None admin] RBAC: Adding query filter params (name=admin) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:45.839954 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a7593a-ccbb-410b-b565-88525561038f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:45.841554 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a7593a-ccbb-410b-b565-88525561038f None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:45.858936 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 305/606] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 08:32:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:45.865026 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a7593a-ccbb-410b-b565-88525561038f None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:45.941444 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:45.957252 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:45.970095 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 302/607] 10.222.0.30 () {58 vars in 1282 bytes} [Sat Oct 15 08:32:45 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 606 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 15 08:32:45.971486 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bf274e-0d00-41fe-9175-ec97fcf37893 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:45.971775 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bf274e-0d00-41fe-9175-ec97fcf37893 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:45.972049 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bf274e-0d00-41fe-9175-ec97fcf37893 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:45.977649 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 303/608] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:32:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:32:45.978470 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bf274e-0d00-41fe-9175-ec97fcf37893 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:46.009029 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a7593a-ccbb-410b-b565-88525561038f None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:46.051035 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d64785-df0f-458c-b4be-56ff3ed2dabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:46.051743 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d64785-df0f-458c-b4be-56ff3ed2dabd None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:46.052052 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d64785-df0f-458c-b4be-56ff3ed2dabd None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:46.052335 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d64785-df0f-458c-b4be-56ff3ed2dabd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:46.066579 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d64785-df0f-458c-b4be-56ff3ed2dabd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:46.069049 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d64785-df0f-458c-b4be-56ff3ed2dabd None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:46.087318 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-87d64785-df0f-458c-b4be-56ff3ed2dabd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:46.087318 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:46.087318 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:46.087318 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:46.087318 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:46.087318 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:46.087318 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:46.087318 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:46.087318 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:46.087318 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:46.087318 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:46.087318 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:46.087318 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 08:32:46.087318 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 08:32:46.087318 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 08:32:46.087318 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 08:32:46.088425 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:46.088425 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:46.088425 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:46.088425 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:46.088425 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:46.088425 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:46.088425 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:46.088425 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:46.088425 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:46.088425 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:46.088425 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:46.088425 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:46.088425 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:46.088425 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:46.088425 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:46.088425 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:46.088425 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 08:32:46.089339 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 08:32:46.089339 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:46.089339 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:46.089339 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 306/609] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:32:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:46.090096 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a9427-93b0-4ea0-8225-f546fbca162e None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:46.090382 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a9427-93b0-4ea0-8225-f546fbca162e None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:46.090657 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a9427-93b0-4ea0-8225-f546fbca162e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:46.109502 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bf274e-0d00-41fe-9175-ec97fcf37893 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:46.111253 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-204a9427-93b0-4ea0-8225-f546fbca162e None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:32:46.113394 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-204a9427-93b0-4ea0-8225-f546fbca162e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:32:46.151308 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabc69f6-74b6-4d20-a15c-d722f52f48ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:46.152038 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabc69f6-74b6-4d20-a15c-d722f52f48ab None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:46.152302 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabc69f6-74b6-4d20-a15c-d722f52f48ab None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:46.152588 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabc69f6-74b6-4d20-a15c-d722f52f48ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:46.163542 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabc69f6-74b6-4d20-a15c-d722f52f48ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:46.165605 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabc69f6-74b6-4d20-a15c-d722f52f48ab None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:46.181056 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-dabc69f6-74b6-4d20-a15c-d722f52f48ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:46.181056 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:46.181056 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:46.181056 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:46.181056 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:46.181056 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:46.181056 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:46.181056 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:46.181056 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:46.181056 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:46.181056 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:46.181056 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:46.181056 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 08:32:46.181056 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 08:32:46.181056 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 08:32:46.181056 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 08:32:46.182315 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:46.182315 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:46.182315 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:46.182315 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:46.182315 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:46.182315 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:46.182315 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:46.182315 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:46.182315 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:46.182315 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:46.182315 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:46.182315 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:46.182315 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:46.182315 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:46.182315 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:46.182315 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:46.182315 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 08:32:46.183426 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 08:32:46.183426 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:46.183426 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:46.183426 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 304/610] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:32:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:46.183426 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabc69f6-74b6-4d20-a15c-d722f52f48ab None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:46.200335 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 307/611] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:32:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 08:32:46.201853 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e31527-8768-433b-b44f-cad9333a83bc None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:46.201853 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e31527-8768-433b-b44f-cad9333a83bc None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:46.202181 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e31527-8768-433b-b44f-cad9333a83bc None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:46.203790 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 308/612] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:32:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:32:46.204573 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e31527-8768-433b-b44f-cad9333a83bc None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:46.233095 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:46.255757 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:46.309561 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabc69f6-74b6-4d20-a15c-d722f52f48ab None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:46.331572 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e31527-8768-433b-b44f-cad9333a83bc None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:46.354403 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f429ff-3745-412b-8243-808773b33a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:46.355178 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f429ff-3745-412b-8243-808773b33a43 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:46.355554 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f429ff-3745-412b-8243-808773b33a43 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:46.356441 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f429ff-3745-412b-8243-808773b33a43 None admin] PATH_INFO: `/v3/domains` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:46.357407 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f429ff-3745-412b-8243-808773b33a43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:46.357921 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f429ff-3745-412b-8243-808773b33a43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:46.360022 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f429ff-3745-412b-8243-808773b33a43 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:46.371922 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e43439-bd90-45b5-8140-6dd58ff47d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:46.373251 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e43439-bd90-45b5-8140-6dd58ff47d33 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:46.373615 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e43439-bd90-45b5-8140-6dd58ff47d33 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:46.374129 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e43439-bd90-45b5-8140-6dd58ff47d33 None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:46.375013 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e43439-bd90-45b5-8140-6dd58ff47d33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:46.375398 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e43439-bd90-45b5-8140-6dd58ff47d33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:46.376603 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 305/613] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 08:32:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:46.376902 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e43439-bd90-45b5-8140-6dd58ff47d33 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:46.378139 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea27603-0b01-4cef-bbdb-8c5c00f7eef0 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:46.378785 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea27603-0b01-4cef-bbdb-8c5c00f7eef0 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:46.379127 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea27603-0b01-4cef-bbdb-8c5c00f7eef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:46.394265 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 309/614] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 08:32:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:46.395126 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e43439-bd90-45b5-8140-6dd58ff47d33 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:46.408822 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-bea27603-0b01-4cef-bbdb-8c5c00f7eef0 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:32:46.410829 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-bea27603-0b01-4cef-bbdb-8c5c00f7eef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:32:46.498978 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 306/615] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:32:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 08:32:46.506175 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea27603-0b01-4cef-bbdb-8c5c00f7eef0 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:46.519755 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e43439-bd90-45b5-8140-6dd58ff47d33 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:46.521557 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:46.541187 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:46.560742 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa878374-e73b-4b9c-9cb0-6b32e6898464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:46.561568 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa878374-e73b-4b9c-9cb0-6b32e6898464 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:46.562153 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa878374-e73b-4b9c-9cb0-6b32e6898464 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:46.562546 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa878374-e73b-4b9c-9cb0-6b32e6898464 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:46.571609 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa878374-e73b-4b9c-9cb0-6b32e6898464 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:46.573268 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa878374-e73b-4b9c-9cb0-6b32e6898464 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:46.587819 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-fa878374-e73b-4b9c-9cb0-6b32e6898464 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 15 08:32:46.587819 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:46.587819 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:46.587819 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:46.587819 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:46.587819 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:46.587819 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:46.587819 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:46.587819 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:46.587819 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:46.587819 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:46.587819 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:46.587819 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 08:32:46.587819 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 08:32:46.587819 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 08:32:46.587819 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 08:32:46.589014 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:46.589014 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:46.589014 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 08:32:46.589014 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 08:32:46.589014 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 08:32:46.589014 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 08:32:46.589014 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:46.589014 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:46.589014 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:46.589014 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:46.589014 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:46.589014 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:46.589014 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:46.589014 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:46.589014 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:46.589014 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:46.589014 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:46.589014 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:46.590003 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:46.590003 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:46.590003 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 08:32:46.590003 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 08:32:46.590003 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 08:32:46.590003 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 08:32:46.590003 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 08:32:46.590003 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 08:32:46.590003 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 15 08:32:46.590003 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:46.590003 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 310/616] 10.222.0.30 () {56 vars in 1096 bytes} [Sat Oct 15 08:32:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:46.591263 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa878374-e73b-4b9c-9cb0-6b32e6898464 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:46.661711 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea27603-0b01-4cef-bbdb-8c5c00f7eef0 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:46.713710 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873ec1e2-cf97-4056-9f40-9ea34d8728e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:46.713710 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ec1e2-cf97-4056-9f40-9ea34d8728e1 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:46.713710 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ec1e2-cf97-4056-9f40-9ea34d8728e1 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:46.713710 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ec1e2-cf97-4056-9f40-9ea34d8728e1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:46.721709 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873ec1e2-cf97-4056-9f40-9ea34d8728e1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:46.721709 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873ec1e2-cf97-4056-9f40-9ea34d8728e1 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:46.733338 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-873ec1e2-cf97-4056-9f40-9ea34d8728e1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 08:32:46.733338 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:46.733338 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:46.733338 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:46.733338 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:46.733338 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:46.733338 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:46.733338 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:46.733338 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:46.733338 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:46.733338 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:46.733338 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:46.733338 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 08:32:46.733338 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 08:32:46.733338 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 08:32:46.734322 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 08:32:46.734322 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:46.734322 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:46.734322 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:46.734322 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:46.734322 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:46.734322 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:46.734322 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:46.734322 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:46.734322 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:46.734322 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:46.734322 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:46.734322 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:46.734322 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:46.734322 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:46.734322 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:46.734322 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:46.735215 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 08:32:46.735215 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 08:32:46.735215 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 08:32:46.735215 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 08:32:46.735215 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 08:32:46.735215 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 08:32:46.735215 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 08:32:46.735215 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:46.736077 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa878374-e73b-4b9c-9cb0-6b32e6898464 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:46.739400 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 307/617] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 08:32:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:46.740248 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873ec1e2-cf97-4056-9f40-9ea34d8728e1 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:46.776076 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75795992-0e62-4cbe-97a0-7e7daa29136b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:46.776817 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75795992-0e62-4cbe-97a0-7e7daa29136b None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:46.777169 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75795992-0e62-4cbe-97a0-7e7daa29136b None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:46.777542 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75795992-0e62-4cbe-97a0-7e7daa29136b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:46.782242 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75795992-0e62-4cbe-97a0-7e7daa29136b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:46.783702 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75795992-0e62-4cbe-97a0-7e7daa29136b None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:46.793042 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-75795992-0e62-4cbe-97a0-7e7daa29136b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 15 08:32:46.793042 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:46.793042 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 08:32:46.793042 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 08:32:46.793042 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 08:32:46.793042 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 08:32:46.793042 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 08:32:46.793042 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 08:32:46.793042 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 15 08:32:46.793042 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:46.793042 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 08:32:46.793042 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:46.793042 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:46.793042 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:46.793042 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:46.793042 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:46.793042 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:46.794154 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:46.794154 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:46.794154 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:46.794154 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:46.794154 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:46.794154 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:46.794154 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 08:32:46.794154 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 08:32:46.794154 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 08:32:46.794154 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 08:32:46.794154 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:46.794154 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:46.794154 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:46.794154 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:46.794154 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:46.794154 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:46.794999 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:46.794999 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:46.794999 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:46.794999 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:46.794999 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:46.794999 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:46.794999 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:46.794999 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:46.794999 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:46.794999 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:46.794999 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 08:32:46.794999 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 08:32:46.794999 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 15 08:32:46.794999 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:46.797744 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 311/618] 10.222.0.30 () {56 vars in 1129 bytes} [Sat Oct 15 08:32:46 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:46.799140 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6883ed-0c25-4262-bcd3-a58315bf0ada None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:46.799640 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6883ed-0c25-4262-bcd3-a58315bf0ada None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:46.799994 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6883ed-0c25-4262-bcd3-a58315bf0ada None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:46.801474 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 312/619] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:32:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:32:46.802293 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6883ed-0c25-4262-bcd3-a58315bf0ada None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:46.830744 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:46.846073 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:46.856266 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873ec1e2-cf97-4056-9f40-9ea34d8728e1 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:46.891232 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8394a19f-ad54-4262-aae8-2ca6f85b2915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:46.892092 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8394a19f-ad54-4262-aae8-2ca6f85b2915 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:46.892374 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8394a19f-ad54-4262-aae8-2ca6f85b2915 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:46.892647 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8394a19f-ad54-4262-aae8-2ca6f85b2915 None admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:46.893371 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8394a19f-ad54-4262-aae8-2ca6f85b2915 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:46.893698 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8394a19f-ad54-4262-aae8-2ca6f85b2915 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:46.895542 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8394a19f-ad54-4262-aae8-2ca6f85b2915 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:46.911707 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6883ed-0c25-4262-bcd3-a58315bf0ada None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:46.915491 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 308/620] 10.222.0.30 () {56 vars in 1135 bytes} [Sat Oct 15 08:32:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:46.916425 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8394a19f-ad54-4262-aae8-2ca6f85b2915 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:46.952571 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8575b15-2e05-4350-bf90-1c373cc16bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:46.953178 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8575b15-2e05-4350-bf90-1c373cc16bef None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:46.953462 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8575b15-2e05-4350-bf90-1c373cc16bef None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:46.953770 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8575b15-2e05-4350-bf90-1c373cc16bef None admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:46.954494 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8575b15-2e05-4350-bf90-1c373cc16bef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:46.954804 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8575b15-2e05-4350-bf90-1c373cc16bef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:46.956232 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8575b15-2e05-4350-bf90-1c373cc16bef None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:46.973365 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 313/621] 10.222.0.30 () {56 vars in 1148 bytes} [Sat Oct 15 08:32:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:46.973365 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8575b15-2e05-4350-bf90-1c373cc16bef None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:47.030999 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8394a19f-ad54-4262-aae8-2ca6f85b2915 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:47.057051 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:47.064665 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b83d02-67d3-48ea-b17d-c730c46fbf15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:47.065271 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b83d02-67d3-48ea-b17d-c730c46fbf15 None admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:47.065556 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b83d02-67d3-48ea-b17d-c730c46fbf15 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:47.065867 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b83d02-67d3-48ea-b17d-c730c46fbf15 None admin] PATH_INFO: `/v3/services` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:47.066513 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b83d02-67d3-48ea-b17d-c730c46fbf15 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:47.067898 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b83d02-67d3-48ea-b17d-c730c46fbf15 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:47.072326 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8575b15-2e05-4350-bf90-1c373cc16bef None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:47.084379 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 309/622] 10.222.0.30 () {60 vars in 1141 bytes} [Sat Oct 15 08:32:46 2022] POST /identity/v3/services => generated 269 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:32:47.085626 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c455f2b6-318e-42d5-9255-f465dc4873ab None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:47.085968 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c455f2b6-318e-42d5-9255-f465dc4873ab None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:47.086956 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c455f2b6-318e-42d5-9255-f465dc4873ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:47.106797 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-c455f2b6-318e-42d5-9255-f465dc4873ab None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:32:47.107958 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-c455f2b6-318e-42d5-9255-f465dc4873ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:32:47.111374 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:47.124761 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73001aff-a305-4b98-b78e-ae2416a0b003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:47.125376 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73001aff-a305-4b98-b78e-ae2416a0b003 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:47.125657 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73001aff-a305-4b98-b78e-ae2416a0b003 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:47.125965 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73001aff-a305-4b98-b78e-ae2416a0b003 None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:47.126604 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73001aff-a305-4b98-b78e-ae2416a0b003 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:47.126905 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73001aff-a305-4b98-b78e-ae2416a0b003 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:47.128285 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73001aff-a305-4b98-b78e-ae2416a0b003 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:47.133881 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 314/623] 10.222.0.30 () {56 vars in 1124 bytes} [Sat Oct 15 08:32:46 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:47.134706 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73001aff-a305-4b98-b78e-ae2416a0b003 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:47.196320 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 310/624] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:32:47 2022] POST /identity/v3/auth/tokens => generated 1112 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 08:32:47.197336 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c455f2b6-318e-42d5-9255-f465dc4873ab None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:47.270746 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73001aff-a305-4b98-b78e-ae2416a0b003 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:47.314926 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3565ed4-aed4-4883-a673-bea1a64ea7a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:47.315546 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3565ed4-aed4-4883-a673-bea1a64ea7a7 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:47.315831 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3565ed4-aed4-4883-a673-bea1a64ea7a7 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:47.316110 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3565ed4-aed4-4883-a673-bea1a64ea7a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:47.316919 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c455f2b6-318e-42d5-9255-f465dc4873ab None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:47.332846 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:47.376655 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9a23c2-3923-43b2-bce4-58396552e995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:47.377450 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a23c2-3923-43b2-bce4-58396552e995 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:47.377761 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a23c2-3923-43b2-bce4-58396552e995 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:47.378042 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a23c2-3923-43b2-bce4-58396552e995 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:47.379216 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9a23c2-3923-43b2-bce4-58396552e995 None admin] RBAC: Adding query filter params (role.id=b1b0b29beb794324826a4c4003ae293b, scope.project.id=5dfceb59b3794439a4586b3cbe408f0e, user.id=c5b2a0e623c348ad94d4820080677be6) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:47.379582 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9a23c2-3923-43b2-bce4-58396552e995 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:47.381038 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9a23c2-3923-43b2-bce4-58396552e995 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:47.391412 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 311/625] 10.222.0.30 () {56 vars in 1374 bytes} [Sat Oct 15 08:32:47 2022] GET /identity/v3/role_assignments?user.id=c5b2a0e623c348ad94d4820080677be6&scope.project.id=5dfceb59b3794439a4586b3cbe408f0e&role.id=b1b0b29beb794324826a4c4003ae293b => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:47.392521 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9a23c2-3923-43b2-bce4-58396552e995 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:47.406123 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:47.415450 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3565ed4-aed4-4883-a673-bea1a64ea7a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:47.417369 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3565ed4-aed4-4883-a673-bea1a64ea7a7 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:47.545311 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 315/626] 10.222.0.30 () {58 vars in 1282 bytes} [Sat Oct 15 08:32:47 2022] GET /identity/v3/auth/tokens => generated 1112 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 15 08:32:47.551412 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3565ed4-aed4-4883-a673-bea1a64ea7a7 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:47.591697 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9a23c2-3923-43b2-bce4-58396552e995 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:47.607131 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:47.630224 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83468e1b-84cf-440a-bdae-52478d358c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:47.630758 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83468e1b-84cf-440a-bdae-52478d358c85 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:47.631036 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83468e1b-84cf-440a-bdae-52478d358c85 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:47.631306 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83468e1b-84cf-440a-bdae-52478d358c85 None admin] PATH_INFO: `/v3/services` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:47.631932 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83468e1b-84cf-440a-bdae-52478d358c85 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:47.632232 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83468e1b-84cf-440a-bdae-52478d358c85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:47.634189 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83468e1b-84cf-440a-bdae-52478d358c85 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:47.639268 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 312/627] 10.222.0.30 () {56 vars in 1124 bytes} [Sat Oct 15 08:32:47 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:47.640061 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83468e1b-84cf-440a-bdae-52478d358c85 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:47.674669 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3565ed4-aed4-4883-a673-bea1a64ea7a7 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:47.710888 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:47.715908 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6c52cc-bfb7-4339-8780-896eed951009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:47.716613 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c52cc-bfb7-4339-8780-896eed951009 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:47.716889 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c52cc-bfb7-4339-8780-896eed951009 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:47.717159 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c52cc-bfb7-4339-8780-896eed951009 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:47.729052 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6c52cc-bfb7-4339-8780-896eed951009 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:47.730533 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6c52cc-bfb7-4339-8780-896eed951009 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:47.746623 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-de6c52cc-bfb7-4339-8780-896eed951009 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:47.746623 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:47.746623 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:47.746623 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:47.746623 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:47.746623 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:47.746623 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:47.746623 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:47.746623 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:47.746623 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:47.746623 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:47.746623 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:47.746623 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 08:32:47.746623 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 08:32:47.746623 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 08:32:47.746623 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 08:32:47.747648 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:47.747648 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:47.747648 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:47.747648 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:47.747648 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:47.747648 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:47.747648 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:47.747648 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:47.747648 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:47.747648 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:47.747648 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:47.747648 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:47.747648 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:47.747648 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:47.747648 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:47.747648 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:47.747648 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 08:32:47.748550 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 08:32:47.748550 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:47.748550 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:47.748740 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83468e1b-84cf-440a-bdae-52478d358c85 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:47.748815 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 316/628] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:32:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:47.748928 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6c52cc-bfb7-4339-8780-896eed951009 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:47.796477 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adc793d-34df-4ed8-9158-b3ae29ff1d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:47.797109 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc793d-34df-4ed8-9158-b3ae29ff1d55 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:47.797385 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc793d-34df-4ed8-9158-b3ae29ff1d55 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:47.797799 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc793d-34df-4ed8-9158-b3ae29ff1d55 None admin] PATH_INFO: `/v3/services/network` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:47.798383 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adc793d-34df-4ed8-9158-b3ae29ff1d55 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:47.799845 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adc793d-34df-4ed8-9158-b3ae29ff1d55 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:47.811122 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-6adc793d-34df-4ed8-9158-b3ae29ff1d55 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 15 08:32:47.811122 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:47.811122 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 08:32:47.811122 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 08:32:47.811122 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 08:32:47.811122 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 08:32:47.811122 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 08:32:47.811122 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 08:32:47.811122 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 15 08:32:47.811122 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:47.811122 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 08:32:47.811122 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:47.811122 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:47.811122 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:47.811122 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:47.811122 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:47.811122 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:47.812132 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:47.812132 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:47.812132 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:47.812132 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:47.812132 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:47.812132 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:47.812132 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 08:32:47.812132 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 08:32:47.812132 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 08:32:47.812132 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 08:32:47.812132 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:47.812132 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:47.812132 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:47.812132 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:47.812132 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:47.812132 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:47.812984 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:47.812984 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:47.812984 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:47.812984 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:47.812984 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:47.812984 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:47.812984 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:47.812984 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:47.812984 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:47.812984 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:47.812984 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 08:32:47.812984 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 08:32:47.812984 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 15 08:32:47.812984 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:47.812984 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 313/629] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 08:32:47 2022] GET /identity/v3/services/network => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:47.814547 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adc793d-34df-4ed8-9158-b3ae29ff1d55 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:47.865881 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:47.870292 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6c52cc-bfb7-4339-8780-896eed951009 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:47.919109 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b09c0a-e775-4e51-a5d4-5996aad9f5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:47.919962 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b09c0a-e775-4e51-a5d4-5996aad9f5a4 None admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:47.920703 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b09c0a-e775-4e51-a5d4-5996aad9f5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:47.921185 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b09c0a-e775-4e51-a5d4-5996aad9f5a4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:47.922026 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b09c0a-e775-4e51-a5d4-5996aad9f5a4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:47.924228 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b09c0a-e775-4e51-a5d4-5996aad9f5a4 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:47.926299 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adc793d-34df-4ed8-9158-b3ae29ff1d55 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:47.953853 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 317/630] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:32:47 2022] POST /identity/v3/endpoints => generated 346 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:32:47.954915 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b09c0a-e775-4e51-a5d4-5996aad9f5a4 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:47.974304 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:47.996652 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac4a796-d1c6-443d-829a-bbed3e5afe8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:47.997277 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac4a796-d1c6-443d-829a-bbed3e5afe8e None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:47.997553 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac4a796-d1c6-443d-829a-bbed3e5afe8e None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:47.997851 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac4a796-d1c6-443d-829a-bbed3e5afe8e None admin] PATH_INFO: `/v3/domains` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:47.998533 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac4a796-d1c6-443d-829a-bbed3e5afe8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:47.998816 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac4a796-d1c6-443d-829a-bbed3e5afe8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:48.000242 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac4a796-d1c6-443d-829a-bbed3e5afe8e None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:48.028765 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 314/631] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 08:32:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:48.035522 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac4a796-d1c6-443d-829a-bbed3e5afe8e None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:48.154863 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b09c0a-e775-4e51-a5d4-5996aad9f5a4 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:48.194092 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ec5487-fd61-490a-a111-02675fd673bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:48.194627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec5487-fd61-490a-a111-02675fd673bb None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:48.194907 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec5487-fd61-490a-a111-02675fd673bb None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:48.195184 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec5487-fd61-490a-a111-02675fd673bb None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:48.195846 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ec5487-fd61-490a-a111-02675fd673bb None admin] RBAC: Adding query filter params (name=network) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:48.196149 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ec5487-fd61-490a-a111-02675fd673bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:48.197514 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ec5487-fd61-490a-a111-02675fd673bb None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:48.206104 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 318/632] 10.222.0.30 () {56 vars in 1112 bytes} [Sat Oct 15 08:32:47 2022] GET /identity/v3/services?name=network => generated 124 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:48.211631 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ec5487-fd61-490a-a111-02675fd673bb None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:48.229415 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac4a796-d1c6-443d-829a-bbed3e5afe8e None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:48.242119 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:48.284203 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d34173-6957-4fc7-8be4-9fca0e599577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:48.285450 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d34173-6957-4fc7-8be4-9fca0e599577 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:48.286434 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d34173-6957-4fc7-8be4-9fca0e599577 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:48.290004 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d34173-6957-4fc7-8be4-9fca0e599577 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:48.302264 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d34173-6957-4fc7-8be4-9fca0e599577 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:48.303786 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d34173-6957-4fc7-8be4-9fca0e599577 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:48.318329 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-45d34173-6957-4fc7-8be4-9fca0e599577 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 08:32:48.318329 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:48.318329 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:48.318329 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:48.318329 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:48.318329 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:48.318329 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:48.318329 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:48.318329 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:48.318329 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:48.318329 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:48.318329 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:48.318329 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 08:32:48.318329 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 08:32:48.318329 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 08:32:48.319282 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 08:32:48.319282 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:48.319282 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:48.319282 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:48.319282 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:48.319282 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:48.319282 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:48.319282 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:48.319282 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:48.319282 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:48.319282 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:48.319282 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:48.319282 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:48.319282 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:48.319282 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:48.319282 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:48.319282 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:48.320192 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 08:32:48.320192 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 08:32:48.320192 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 08:32:48.320192 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 08:32:48.320192 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 08:32:48.320192 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 08:32:48.320192 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 08:32:48.320192 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:48.320192 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 315/633] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 08:32:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:48.325555 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d34173-6957-4fc7-8be4-9fca0e599577 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:48.362225 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ec5487-fd61-490a-a111-02675fd673bb None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:48.431783 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:48.434440 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef03592-3394-4da6-a17f-67c32307fac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:48.438456 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef03592-3394-4da6-a17f-67c32307fac8 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:48.438739 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef03592-3394-4da6-a17f-67c32307fac8 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:48.439016 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef03592-3394-4da6-a17f-67c32307fac8 None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:48.439721 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef03592-3394-4da6-a17f-67c32307fac8 None admin] RBAC: Adding query filter params (type=network) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:48.440032 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef03592-3394-4da6-a17f-67c32307fac8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:48.441444 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef03592-3394-4da6-a17f-67c32307fac8 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:48.451064 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 319/634] 10.222.0.30 () {56 vars in 1112 bytes} [Sat Oct 15 08:32:48 2022] GET /identity/v3/services?type=network => generated 350 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:48.456281 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef03592-3394-4da6-a17f-67c32307fac8 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:48.569446 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d34173-6957-4fc7-8be4-9fca0e599577 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:48.576526 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:48.603639 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef03592-3394-4da6-a17f-67c32307fac8 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:48.641529 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bac5d8-c8e3-42e0-b4ad-3991538dcb34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:48.642431 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bac5d8-c8e3-42e0-b4ad-3991538dcb34 None admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:48.642711 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bac5d8-c8e3-42e0-b4ad-3991538dcb34 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:48.646332 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bac5d8-c8e3-42e0-b4ad-3991538dcb34 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:48.647035 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bac5d8-c8e3-42e0-b4ad-3991538dcb34 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:48.648457 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bac5d8-c8e3-42e0-b4ad-3991538dcb34 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:48.655427 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d921291-0ab9-4ee4-b59b-e34f280a270e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:48.656045 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d921291-0ab9-4ee4-b59b-e34f280a270e None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:48.656328 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d921291-0ab9-4ee4-b59b-e34f280a270e None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:48.656606 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d921291-0ab9-4ee4-b59b-e34f280a270e None admin] PATH_INFO: `/v3/projects` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:48.657304 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d921291-0ab9-4ee4-b59b-e34f280a270e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:48.657613 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d921291-0ab9-4ee4-b59b-e34f280a270e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:48.659160 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d921291-0ab9-4ee4-b59b-e34f280a270e None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:48.682381 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 316/635] 10.222.0.30 () {56 vars in 1148 bytes} [Sat Oct 15 08:32:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 08:32:48.690679 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d921291-0ab9-4ee4-b59b-e34f280a270e None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:48.694909 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 320/636] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:32:48 2022] POST /identity/v3/endpoints => generated 337 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:32:48.812721 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d921291-0ab9-4ee4-b59b-e34f280a270e None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:48.830092 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:48.880815 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8b37b8-b94d-47be-80e4-c3c7b610220c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:48.881453 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b37b8-b94d-47be-80e4-c3c7b610220c None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:48.881763 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b37b8-b94d-47be-80e4-c3c7b610220c None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:48.882052 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b37b8-b94d-47be-80e4-c3c7b610220c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:48.882894 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8b37b8-b94d-47be-80e4-c3c7b610220c None admin] RBAC: Adding query filter params (role.id=14615e30ed3f4d8cb78ee5e0bf9b5761, scope.project.id=5dfceb59b3794439a4586b3cbe408f0e, user.id=6e3009e20e4c49e5a74f7fd793b2c329) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:48.883744 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8b37b8-b94d-47be-80e4-c3c7b610220c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:48.885268 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8b37b8-b94d-47be-80e4-c3c7b610220c None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:48.892810 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 317/637] 10.222.0.30 () {56 vars in 1374 bytes} [Sat Oct 15 08:32:48 2022] GET /identity/v3/role_assignments?user.id=6e3009e20e4c49e5a74f7fd793b2c329&scope.project.id=5dfceb59b3794439a4586b3cbe408f0e&role.id=14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:49.610359 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c429490-1b28-47e2-8c81-fc3d2c63c3a2 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:49.610880 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c429490-1b28-47e2-8c81-fc3d2c63c3a2 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:49.610960 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c429490-1b28-47e2-8c81-fc3d2c63c3a2 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:49.612468 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 321/638] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:32:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:32:49.618400 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c8b11-1a74-404d-9369-bf2ad63f7463 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:49.618801 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c8b11-1a74-404d-9369-bf2ad63f7463 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:49.618973 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c8b11-1a74-404d-9369-bf2ad63f7463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:49.637879 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-5c3c8b11-1a74-404d-9369-bf2ad63f7463 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:32:49.639063 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-5c3c8b11-1a74-404d-9369-bf2ad63f7463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:32:49.712483 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 318/639] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:32:49 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 08:32:49.785997 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355c22d-27a4-44ab-a954-291b553781b4 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:49.786307 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355c22d-27a4-44ab-a954-291b553781b4 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:49.786559 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355c22d-27a4-44ab-a954-291b553781b4 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:49.788155 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 322/640] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:32:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:32:49.793506 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73704c1-3195-49f6-8bae-d42d1b713db0 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:49.794057 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73704c1-3195-49f6-8bae-d42d1b713db0 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:49.794540 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73704c1-3195-49f6-8bae-d42d1b713db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:49.813552 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-c73704c1-3195-49f6-8bae-d42d1b713db0 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:32:49.815045 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-c73704c1-3195-49f6-8bae-d42d1b713db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:32:49.889904 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 319/641] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:32:49 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 08:32:49.984143 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a355c22d-27a4-44ab-a954-291b553781b4 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:50.090228 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a355c22d-27a4-44ab-a954-291b553781b4 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:50.126492 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b62c0f2-9b38-4c24-9b40-f7aee317d9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:50.127486 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62c0f2-9b38-4c24-9b40-f7aee317d9cc None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:50.127997 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62c0f2-9b38-4c24-9b40-f7aee317d9cc None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:50.128474 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62c0f2-9b38-4c24-9b40-f7aee317d9cc None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:50.129264 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b62c0f2-9b38-4c24-9b40-f7aee317d9cc None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:50.130910 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b62c0f2-9b38-4c24-9b40-f7aee317d9cc None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:50.140612 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-8b62c0f2-9b38-4c24-9b40-f7aee317d9cc None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 15 08:32:50.140612 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:50.140612 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 08:32:50.140612 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 08:32:50.140612 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 08:32:50.140612 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 08:32:50.140612 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 08:32:50.140612 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 08:32:50.140612 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 15 08:32:50.140612 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:50.140612 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 08:32:50.140612 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:50.140612 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:50.140612 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:50.140612 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:50.140612 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:50.141539 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:50.141539 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:50.141539 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:50.141539 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:50.141539 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:50.141539 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:50.141539 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:50.141539 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 08:32:50.141539 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 08:32:50.141539 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 08:32:50.141539 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 08:32:50.141539 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:50.141539 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:50.141539 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:50.141539 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:50.141539 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:50.141539 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:50.142749 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:50.142749 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:50.142749 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:50.142749 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:50.142749 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:50.142749 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:50.142749 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:50.142749 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:50.142749 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:50.142749 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:50.142749 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 08:32:50.142749 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 08:32:50.142749 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 15 08:32:50.142749 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:50.145467 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 323/642] 10.222.0.30 () {56 vars in 1132 bytes} [Sat Oct 15 08:32:49 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:50.151605 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73704c1-3195-49f6-8bae-d42d1b713db0 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:50.160272 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b62c0f2-9b38-4c24-9b40-f7aee317d9cc None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:50.163486 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:50.202031 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:50.265790 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73704c1-3195-49f6-8bae-d42d1b713db0 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:50.269525 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b62c0f2-9b38-4c24-9b40-f7aee317d9cc None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:50.303333 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85867c9d-50ad-44ee-80b1-7544545ecf39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:50.303965 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85867c9d-50ad-44ee-80b1-7544545ecf39 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:50.304245 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85867c9d-50ad-44ee-80b1-7544545ecf39 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:50.304627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85867c9d-50ad-44ee-80b1-7544545ecf39 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:50.305891 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8597c936-6895-4d4a-b57c-f217bf0cb084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:50.306662 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8597c936-6895-4d4a-b57c-f217bf0cb084 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:50.307219 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8597c936-6895-4d4a-b57c-f217bf0cb084 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:50.307633 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8597c936-6895-4d4a-b57c-f217bf0cb084 None admin] PATH_INFO: `/v3/services` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:50.308510 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8597c936-6895-4d4a-b57c-f217bf0cb084 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:50.308963 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8597c936-6895-4d4a-b57c-f217bf0cb084 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:50.310521 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8597c936-6895-4d4a-b57c-f217bf0cb084 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:50.315805 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85867c9d-50ad-44ee-80b1-7544545ecf39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:32:50.316283 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 320/643] 10.222.0.30 () {56 vars in 1126 bytes} [Sat Oct 15 08:32:50 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:50.321446 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8597c936-6895-4d4a-b57c-f217bf0cb084 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:50.326105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-85867c9d-50ad-44ee-80b1-7544545ecf39 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 15 08:32:50.326105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:50.326105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:50.326105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:50.326105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:50.326105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:50.326105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:50.326105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:50.326105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:50.326105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:50.326105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:50.326105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:50.326105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:32:50.326105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:32:50.326105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:32:50.326105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:32:50.327274 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:32:50.327274 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:32:50.327274 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:32:50.327274 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:32:50.327274 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:50.327274 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:50.327274 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:50.327274 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:50.327274 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:50.327274 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:50.327274 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:50.327274 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:50.327274 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:50.327274 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:50.327274 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:50.327274 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:50.332955 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:50.332955 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:50.332955 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:50.332955 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:50.332955 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:32:50.332955 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:32:50.332955 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:32:50.332955 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:32:50.332955 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:32:50.332955 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:32:50.332955 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 15 08:32:50.332955 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:50.332955 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 324/644] 10.222.0.30 () {56 vars in 1120 bytes} [Sat Oct 15 08:32:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 08:32:50.335671 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85867c9d-50ad-44ee-80b1-7544545ecf39 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:50.429111 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:50.441703 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8597c936-6895-4d4a-b57c-f217bf0cb084 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:50.442447 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:50.488326 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b23b16-483f-458f-ac3d-2d9e3a1de32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:50.489049 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b23b16-483f-458f-ac3d-2d9e3a1de32f None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:50.489331 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b23b16-483f-458f-ac3d-2d9e3a1de32f None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:50.489755 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b23b16-483f-458f-ac3d-2d9e3a1de32f None admin] PATH_INFO: `/v3/services` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:50.490413 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b23b16-483f-458f-ac3d-2d9e3a1de32f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:50.490715 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b23b16-483f-458f-ac3d-2d9e3a1de32f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:50.493588 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b23b16-483f-458f-ac3d-2d9e3a1de32f None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:50.497644 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 321/645] 10.222.0.30 () {56 vars in 1126 bytes} [Sat Oct 15 08:32:50 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 08:32:50.498025 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85867c9d-50ad-44ee-80b1-7544545ecf39 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:50.498991 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3929fa-4e7e-4bc8-8157-92f0390f4ee8 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:50.499281 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3929fa-4e7e-4bc8-8157-92f0390f4ee8 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:50.499557 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3929fa-4e7e-4bc8-8157-92f0390f4ee8 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:50.500747 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 322/646] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:32:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:32:50.505374 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3929fa-4e7e-4bc8-8157-92f0390f4ee8 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:50.540320 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be0fcb3-7f45-4fb0-8e2d-94fdd4b40baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:50.541064 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0fcb3-7f45-4fb0-8e2d-94fdd4b40baa None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:50.541350 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0fcb3-7f45-4fb0-8e2d-94fdd4b40baa None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:50.541623 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0fcb3-7f45-4fb0-8e2d-94fdd4b40baa None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:50.542432 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be0fcb3-7f45-4fb0-8e2d-94fdd4b40baa None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:50.542851 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be0fcb3-7f45-4fb0-8e2d-94fdd4b40baa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:50.544207 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be0fcb3-7f45-4fb0-8e2d-94fdd4b40baa None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:50.556713 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 325/647] 10.222.0.30 () {56 vars in 1115 bytes} [Sat Oct 15 08:32:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 08:32:50.558292 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ce5c56-c76e-4756-940e-dd84df4757e1 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:50.558643 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ce5c56-c76e-4756-940e-dd84df4757e1 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:50.558933 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ce5c56-c76e-4756-940e-dd84df4757e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:50.577638 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-22ce5c56-c76e-4756-940e-dd84df4757e1 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:32:50.578850 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-22ce5c56-c76e-4756-940e-dd84df4757e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:32:50.616550 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3929fa-4e7e-4bc8-8157-92f0390f4ee8 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:50.650444 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 326/648] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:32:50 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 08:32:50.654285 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ce5c56-c76e-4756-940e-dd84df4757e1 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:50.660023 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea010560-f832-428c-86fd-340027e21714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:50.661170 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea010560-f832-428c-86fd-340027e21714 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:50.662119 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea010560-f832-428c-86fd-340027e21714 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:50.662546 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea010560-f832-428c-86fd-340027e21714 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:50.663834 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea010560-f832-428c-86fd-340027e21714 None admin] RBAC: Adding query filter params (interface=public, service_id=259011d7f1a14f86aa1efccde13876d5, region_id=RegionOne) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:50.665140 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea010560-f832-428c-86fd-340027e21714 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:50.667019 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea010560-f832-428c-86fd-340027e21714 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:50.675039 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 323/649] 10.222.0.30 () {56 vars in 1251 bytes} [Sat Oct 15 08:32:50 2022] GET /identity/v3/endpoints?service_id=259011d7f1a14f86aa1efccde13876d5&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:50.679747 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea010560-f832-428c-86fd-340027e21714 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:50.712371 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:50.714853 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:50.764281 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ce5c56-c76e-4756-940e-dd84df4757e1 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:50.802513 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dad2f1-3982-4e90-9a65-3ed8e041609a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:50.803502 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dad2f1-3982-4e90-9a65-3ed8e041609a None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:50.803980 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dad2f1-3982-4e90-9a65-3ed8e041609a None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:50.804547 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dad2f1-3982-4e90-9a65-3ed8e041609a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:50.811705 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea010560-f832-428c-86fd-340027e21714 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:50.816330 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dad2f1-3982-4e90-9a65-3ed8e041609a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:50.818015 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dad2f1-3982-4e90-9a65-3ed8e041609a None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:50.834584 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-16dad2f1-3982-4e90-9a65-3ed8e041609a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:50.834584 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:50.834584 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:50.834584 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:50.834584 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:50.834584 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:50.834584 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:50.834584 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:50.834584 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:50.834584 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:50.834584 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:50.834584 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:50.834584 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 08:32:50.834584 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 08:32:50.834584 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 08:32:50.834584 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 08:32:50.835618 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:50.835618 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:50.835618 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:50.835618 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:50.835618 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:50.835618 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:50.835618 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:50.835618 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:50.835618 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:50.835618 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:50.835618 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:50.835618 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:50.835618 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:50.835618 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:50.835618 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:50.835618 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:50.835618 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 08:32:50.836640 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 08:32:50.836640 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:50.836640 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:50.838873 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 327/650] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:32:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:50.844991 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dad2f1-3982-4e90-9a65-3ed8e041609a None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:50.856569 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac26a72-24e6-4cf7-900f-507c08671aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:50.857908 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac26a72-24e6-4cf7-900f-507c08671aa5 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:50.858510 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac26a72-24e6-4cf7-900f-507c08671aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:50.859054 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac26a72-24e6-4cf7-900f-507c08671aa5 None admin] PATH_INFO: `/v3/services` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:50.859862 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac26a72-24e6-4cf7-900f-507c08671aa5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:50.861538 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac26a72-24e6-4cf7-900f-507c08671aa5 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:50.868110 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 324/651] 10.222.0.30 () {56 vars in 1087 bytes} [Sat Oct 15 08:32:50 2022] GET /identity/v3/services => generated 1029 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 08:32:50.930919 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac26a72-24e6-4cf7-900f-507c08671aa5 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:50.974118 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dad2f1-3982-4e90-9a65-3ed8e041609a None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:50.987111 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:51.019871 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188ebefa-3216-4482-a353-fb5ff2940740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:51.020867 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188ebefa-3216-4482-a353-fb5ff2940740 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:51.021346 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188ebefa-3216-4482-a353-fb5ff2940740 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:51.021878 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188ebefa-3216-4482-a353-fb5ff2940740 None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:51.022735 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188ebefa-3216-4482-a353-fb5ff2940740 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:51.023250 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188ebefa-3216-4482-a353-fb5ff2940740 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:51.025422 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188ebefa-3216-4482-a353-fb5ff2940740 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:51.038039 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 328/652] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 08:32:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:51.044572 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188ebefa-3216-4482-a353-fb5ff2940740 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:51.044751 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:51.066903 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac26a72-24e6-4cf7-900f-507c08671aa5 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:51.109647 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f184b0-6803-4481-9934-3639640d778f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:51.110346 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f184b0-6803-4481-9934-3639640d778f None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:51.110631 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f184b0-6803-4481-9934-3639640d778f None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:51.110905 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f184b0-6803-4481-9934-3639640d778f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:51.111508 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f184b0-6803-4481-9934-3639640d778f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:51.113058 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f184b0-6803-4481-9934-3639640d778f None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:51.123751 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-62f184b0-6803-4481-9934-3639640d778f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 15 08:32:51.123751 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:51.123751 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 08:32:51.123751 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 08:32:51.123751 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 08:32:51.123751 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 08:32:51.123751 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 08:32:51.123751 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 08:32:51.123751 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 15 08:32:51.123751 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:51.123751 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 08:32:51.123751 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:51.123751 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:51.123751 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:51.123751 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:51.123751 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:51.123751 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:51.124848 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:51.124848 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:51.124848 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:51.124848 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:51.124848 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:51.124848 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:51.124848 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 08:32:51.124848 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 08:32:51.124848 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 08:32:51.124848 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 08:32:51.124848 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:51.124848 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:51.124848 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:51.124848 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:51.124848 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:51.124848 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:51.125760 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:51.125760 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:51.125760 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:51.125760 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:51.125760 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:51.125760 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:51.125760 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:51.125760 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:51.125760 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:51.125760 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:51.125760 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 08:32:51.125760 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 08:32:51.125760 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 15 08:32:51.125760 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:51.125760 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 325/653] 10.222.0.30 () {56 vars in 1114 bytes} [Sat Oct 15 08:32:50 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:51.131729 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f184b0-6803-4481-9934-3639640d778f None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:51.178161 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188ebefa-3216-4482-a353-fb5ff2940740 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:51.221707 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7925e3-8d71-469b-8d13-3a1ec33ac194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:51.222717 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7925e3-8d71-469b-8d13-3a1ec33ac194 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:51.223002 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7925e3-8d71-469b-8d13-3a1ec33ac194 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:51.223281 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7925e3-8d71-469b-8d13-3a1ec33ac194 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:51.232017 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7925e3-8d71-469b-8d13-3a1ec33ac194 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:51.233843 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7925e3-8d71-469b-8d13-3a1ec33ac194 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:51.245311 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-5d7925e3-8d71-469b-8d13-3a1ec33ac194 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 15 08:32:51.245311 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:51.245311 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:51.245311 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:51.245311 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:51.245311 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:51.245311 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:51.245311 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:51.245311 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:51.245311 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:51.245311 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:51.245311 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:51.245311 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 08:32:51.245311 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 08:32:51.245311 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 08:32:51.245311 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 08:32:51.246454 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:51.246454 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:51.246454 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 08:32:51.246454 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 08:32:51.246454 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 08:32:51.246454 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 08:32:51.246454 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:51.246454 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:51.246454 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:51.246454 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:51.246454 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:51.246454 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:51.246454 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:51.246454 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:51.246454 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:51.246454 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:51.246454 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:51.246454 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:51.247587 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:51.247587 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:51.247587 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 08:32:51.247587 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 08:32:51.247587 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 08:32:51.247587 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 08:32:51.247587 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 08:32:51.247587 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 08:32:51.247587 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 15 08:32:51.247587 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:51.249833 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 329/654] 10.222.0.30 () {56 vars in 1117 bytes} [Sat Oct 15 08:32:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:51.251153 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f184b0-6803-4481-9934-3639640d778f None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:51.251542 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d3b0a8-ff68-48b2-8c58-783c55f0951b None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:51.251899 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d3b0a8-ff68-48b2-8c58-783c55f0951b None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:51.252245 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d3b0a8-ff68-48b2-8c58-783c55f0951b None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:51.253779 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 330/655] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:32:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:32:51.257868 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d3b0a8-ff68-48b2-8c58-783c55f0951b None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:51.284930 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:51.289952 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db6e464-12a4-4be5-a68d-59e7eecdfbca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:51.290560 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db6e464-12a4-4be5-a68d-59e7eecdfbca None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:51.290837 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db6e464-12a4-4be5-a68d-59e7eecdfbca None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:51.291204 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db6e464-12a4-4be5-a68d-59e7eecdfbca None admin] PATH_INFO: `/v3/services` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:51.291729 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db6e464-12a4-4be5-a68d-59e7eecdfbca None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:51.292031 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db6e464-12a4-4be5-a68d-59e7eecdfbca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:51.293385 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db6e464-12a4-4be5-a68d-59e7eecdfbca None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:51.301725 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 326/656] 10.222.0.30 () {56 vars in 1114 bytes} [Sat Oct 15 08:32:51 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:51.301725 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d236a23d-3f74-4e1c-8c0a-f4059fc1aabc None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:51.301725 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d236a23d-3f74-4e1c-8c0a-f4059fc1aabc None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:51.301725 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d236a23d-3f74-4e1c-8c0a-f4059fc1aabc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:51.318221 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-d236a23d-3f74-4e1c-8c0a-f4059fc1aabc None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:32:51.319325 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-d236a23d-3f74-4e1c-8c0a-f4059fc1aabc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:32:51.327579 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:51.369512 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d3b0a8-ff68-48b2-8c58-783c55f0951b None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:51.400673 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 327/657] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:32:51 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 08:32:51.402689 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d236a23d-3f74-4e1c-8c0a-f4059fc1aabc None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:51.413103 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a69d9a-0864-486d-acf1-c5ff4b861202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:51.413408 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a69d9a-0864-486d-acf1-c5ff4b861202 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:51.413710 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a69d9a-0864-486d-acf1-c5ff4b861202 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:51.414514 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a69d9a-0864-486d-acf1-c5ff4b861202 None admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:51.415283 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a69d9a-0864-486d-acf1-c5ff4b861202 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:51.415591 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a69d9a-0864-486d-acf1-c5ff4b861202 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:51.417266 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a69d9a-0864-486d-acf1-c5ff4b861202 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:51.434739 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 331/658] 10.222.0.30 () {56 vars in 1149 bytes} [Sat Oct 15 08:32:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:51.442747 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a69d9a-0864-486d-acf1-c5ff4b861202 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:51.524795 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d236a23d-3f74-4e1c-8c0a-f4059fc1aabc None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:51.545571 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:51.561042 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9291e5f-246e-4eb2-9b1b-cc23f0606b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:51.562000 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9291e5f-246e-4eb2-9b1b-cc23f0606b1f None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:51.562647 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9291e5f-246e-4eb2-9b1b-cc23f0606b1f None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:51.563123 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9291e5f-246e-4eb2-9b1b-cc23f0606b1f None admin] PATH_INFO: `/v3/services` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:51.563945 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9291e5f-246e-4eb2-9b1b-cc23f0606b1f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:51.564441 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9291e5f-246e-4eb2-9b1b-cc23f0606b1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:51.565964 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a69d9a-0864-486d-acf1-c5ff4b861202 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:51.566294 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9291e5f-246e-4eb2-9b1b-cc23f0606b1f None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:51.572024 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 328/659] 10.222.0.30 () {56 vars in 1114 bytes} [Sat Oct 15 08:32:51 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:51.606694 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afdb87c-9738-41d2-91f8-47a9ff39fbfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:51.607778 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afdb87c-9738-41d2-91f8-47a9ff39fbfd None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:51.608284 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afdb87c-9738-41d2-91f8-47a9ff39fbfd None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:51.608760 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afdb87c-9738-41d2-91f8-47a9ff39fbfd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:51.621192 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afdb87c-9738-41d2-91f8-47a9ff39fbfd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:51.623108 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afdb87c-9738-41d2-91f8-47a9ff39fbfd None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:51.640188 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-7afdb87c-9738-41d2-91f8-47a9ff39fbfd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:51.640188 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:51.640188 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:51.640188 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:51.640188 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:51.640188 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:51.640188 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:51.640188 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:51.640188 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:51.640188 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:51.640188 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:51.640188 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:51.640188 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 08:32:51.640188 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 08:32:51.640188 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 08:32:51.640188 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 08:32:51.641233 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:51.641233 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:51.641233 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:51.641233 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:51.641233 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:51.641233 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:51.641233 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:51.641233 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:51.641233 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:51.641233 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:51.641233 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:51.641233 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:51.641233 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:51.641233 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:51.641233 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:51.641233 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:51.641233 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 08:32:51.642464 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 08:32:51.642464 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:51.642464 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:51.644813 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 332/660] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:32:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:51.653237 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9291e5f-246e-4eb2-9b1b-cc23f0606b1f None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:51.660484 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:51.677703 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afdb87c-9738-41d2-91f8-47a9ff39fbfd None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:51.795522 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9291e5f-246e-4eb2-9b1b-cc23f0606b1f None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:51.808883 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afdb87c-9738-41d2-91f8-47a9ff39fbfd None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:51.845398 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f831ba-f6b0-4798-a2fe-3c83f1de0869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:51.846348 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f831ba-f6b0-4798-a2fe-3c83f1de0869 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:51.846829 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f831ba-f6b0-4798-a2fe-3c83f1de0869 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:51.847325 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f831ba-f6b0-4798-a2fe-3c83f1de0869 None admin] PATH_INFO: `/v3/domains` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:51.848156 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f831ba-f6b0-4798-a2fe-3c83f1de0869 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:51.848654 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f831ba-f6b0-4798-a2fe-3c83f1de0869 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:51.850258 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f831ba-f6b0-4798-a2fe-3c83f1de0869 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:51.853378 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7658de-7578-44b6-aa36-44a16953580f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:51.854061 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7658de-7578-44b6-aa36-44a16953580f None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:51.854347 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7658de-7578-44b6-aa36-44a16953580f None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:51.854624 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7658de-7578-44b6-aa36-44a16953580f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:51.855214 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7658de-7578-44b6-aa36-44a16953580f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:51.856565 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7658de-7578-44b6-aa36-44a16953580f None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:51.859277 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:51.865422 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 329/661] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 08:32:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:51.873730 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-ea7658de-7578-44b6-aa36-44a16953580f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 15 08:32:51.873730 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:51.873730 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 08:32:51.873730 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 08:32:51.873730 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 08:32:51.873730 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 08:32:51.873730 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 08:32:51.873730 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 08:32:51.873730 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 15 08:32:51.873730 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:51.873730 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 08:32:51.873730 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:51.873730 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:51.873730 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:51.873730 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:51.873730 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:51.873730 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:51.874755 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:51.874755 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:51.874755 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:51.874755 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:51.874755 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:51.874755 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:51.874755 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 08:32:51.874755 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 08:32:51.874755 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 08:32:51.874755 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 08:32:51.874755 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:51.874755 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:51.874755 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:51.874755 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:51.874755 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:51.874755 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:51.875612 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:51.875612 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:51.875612 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:51.875612 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:51.875612 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:51.875612 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:51.875612 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:51.875612 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:51.875612 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:51.875612 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:51.875612 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 08:32:51.875612 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 08:32:51.875612 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 15 08:32:51.875612 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:51.876545 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f831ba-f6b0-4798-a2fe-3c83f1de0869 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:51.876631 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 333/662] 10.222.0.30 () {56 vars in 1126 bytes} [Sat Oct 15 08:32:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 08:32:51.878181 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7658de-7578-44b6-aa36-44a16953580f None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:51.957360 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:51.980321 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f831ba-f6b0-4798-a2fe-3c83f1de0869 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:52.008664 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7658de-7578-44b6-aa36-44a16953580f None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:52.024547 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a41341-526a-4c0f-8525-6f24f8f78e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:52.025828 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a41341-526a-4c0f-8525-6f24f8f78e86 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:52.026234 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a41341-526a-4c0f-8525-6f24f8f78e86 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:52.026619 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a41341-526a-4c0f-8525-6f24f8f78e86 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:52.035612 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a41341-526a-4c0f-8525-6f24f8f78e86 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:52.037911 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a41341-526a-4c0f-8525-6f24f8f78e86 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:52.048945 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178f63a4-f039-462f-98a8-55b8289c50fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:52.049810 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f63a4-f039-462f-98a8-55b8289c50fb None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:52.050239 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f63a4-f039-462f-98a8-55b8289c50fb None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:52.050759 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f63a4-f039-462f-98a8-55b8289c50fb None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:52.051535 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178f63a4-f039-462f-98a8-55b8289c50fb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:52.051990 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178f63a4-f039-462f-98a8-55b8289c50fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:52.053291 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-37a41341-526a-4c0f-8525-6f24f8f78e86 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 08:32:52.053291 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:52.053291 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:52.053291 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:52.053291 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:52.053291 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:52.053291 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:52.053291 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:52.053291 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:52.053291 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:52.053291 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:52.053291 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:52.053291 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 08:32:52.053291 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 08:32:52.053291 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 08:32:52.054363 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 08:32:52.054363 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:52.054363 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:52.054363 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:52.054363 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:52.054363 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:52.054363 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:52.054363 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:52.054363 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:52.054363 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:52.054363 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:52.054363 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:52.054363 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:52.054363 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:52.054363 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:52.054363 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:52.054363 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:52.056008 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 08:32:52.056008 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 08:32:52.056008 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 08:32:52.056008 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 08:32:52.056008 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 08:32:52.056008 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 08:32:52.056008 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 08:32:52.056008 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:52.057250 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178f63a4-f039-462f-98a8-55b8289c50fb None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:52.062300 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 330/663] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 08:32:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:52.063316 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a41341-526a-4c0f-8525-6f24f8f78e86 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:52.069395 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 334/664] 10.222.0.30 () {56 vars in 1122 bytes} [Sat Oct 15 08:32:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:52.075212 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178f63a4-f039-462f-98a8-55b8289c50fb None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:52.155415 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:52.177311 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a41341-526a-4c0f-8525-6f24f8f78e86 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:52.205668 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178f63a4-f039-462f-98a8-55b8289c50fb None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:52.213821 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:52.226011 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292fa3f5-f17f-4b14-9007-843bb3eddf08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:52.227321 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292fa3f5-f17f-4b14-9007-843bb3eddf08 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:52.227731 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292fa3f5-f17f-4b14-9007-843bb3eddf08 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:52.228102 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292fa3f5-f17f-4b14-9007-843bb3eddf08 None admin] PATH_INFO: `/v3/projects` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:52.228935 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292fa3f5-f17f-4b14-9007-843bb3eddf08 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:52.229343 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292fa3f5-f17f-4b14-9007-843bb3eddf08 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:52.231081 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292fa3f5-f17f-4b14-9007-843bb3eddf08 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:52.250027 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 331/665] 10.222.0.30 () {56 vars in 1148 bytes} [Sat Oct 15 08:32:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:52.250890 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292fa3f5-f17f-4b14-9007-843bb3eddf08 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:52.251740 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2ea0c7-3c23-470f-b32f-b0c4a751591f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:52.252594 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2ea0c7-3c23-470f-b32f-b0c4a751591f None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:52.253075 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2ea0c7-3c23-470f-b32f-b0c4a751591f None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:52.253860 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2ea0c7-3c23-470f-b32f-b0c4a751591f None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:52.255629 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2ea0c7-3c23-470f-b32f-b0c4a751591f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:52.256167 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2ea0c7-3c23-470f-b32f-b0c4a751591f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:52.258381 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2ea0c7-3c23-470f-b32f-b0c4a751591f None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:52.265033 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 335/666] 10.222.0.30 () {56 vars in 1122 bytes} [Sat Oct 15 08:32:52 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 08:32:52.367305 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292fa3f5-f17f-4b14-9007-843bb3eddf08 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:52.407566 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4726bfba-c609-4e0c-a284-58a05f6f0186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:52.408625 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4726bfba-c609-4e0c-a284-58a05f6f0186 None admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:52.409145 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4726bfba-c609-4e0c-a284-58a05f6f0186 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:52.409625 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4726bfba-c609-4e0c-a284-58a05f6f0186 None admin] PATH_INFO: `/v3/projects/5dfceb59b3794439a4586b3cbe408f0e/users/c5b2a0e623c348ad94d4820080677be6/roles/3e2674a06b1e46739ff0be6a5e20003e` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:52.412325 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4726bfba-c609-4e0c-a284-58a05f6f0186 None admin] RBAC: Authorizing `identity:create_grant(project_id=5dfceb59b3794439a4586b3cbe408f0e, user_id=c5b2a0e623c348ad94d4820080677be6, role_id=3e2674a06b1e46739ff0be6a5e20003e)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:52.414633 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4726bfba-c609-4e0c-a284-58a05f6f0186 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:52.431150 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 332/667] 10.222.0.30 () {58 vars in 1439 bytes} [Sat Oct 15 08:32:52 2022] PUT /identity/v3/projects/5dfceb59b3794439a4586b3cbe408f0e/users/c5b2a0e623c348ad94d4820080677be6/roles/3e2674a06b1e46739ff0be6a5e20003e => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:32:53.071685 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f923cb-a3d7-4c33-b758-89a28c112a67 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:53.072235 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f923cb-a3d7-4c33-b758-89a28c112a67 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:53.072235 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f923cb-a3d7-4c33-b758-89a28c112a67 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:53.073815 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 336/668] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:32:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:32:53.079206 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe44d68-cf60-4a69-adec-eecc7d1d7308 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:53.079598 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe44d68-cf60-4a69-adec-eecc7d1d7308 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:53.079762 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe44d68-cf60-4a69-adec-eecc7d1d7308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:53.098329 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-3fe44d68-cf60-4a69-adec-eecc7d1d7308 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:32:53.099517 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-3fe44d68-cf60-4a69-adec-eecc7d1d7308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:32:53.145151 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:53.178971 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 333/669] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:32:53 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 08:32:53.442626 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f923cb-a3d7-4c33-b758-89a28c112a67 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:53.548878 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f923cb-a3d7-4c33-b758-89a28c112a67 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:53.576611 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:53.584400 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5a1685-98fd-4b69-8a99-ba251854312c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:53.585080 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a1685-98fd-4b69-8a99-ba251854312c None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:53.585751 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a1685-98fd-4b69-8a99-ba251854312c None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:53.586062 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a1685-98fd-4b69-8a99-ba251854312c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:53.586654 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5a1685-98fd-4b69-8a99-ba251854312c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:53.588380 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5a1685-98fd-4b69-8a99-ba251854312c None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:53.597348 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-6f5a1685-98fd-4b69-8a99-ba251854312c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 15 08:32:53.597348 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:53.597348 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 08:32:53.597348 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 08:32:53.597348 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 08:32:53.597348 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 08:32:53.597348 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 08:32:53.597348 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 08:32:53.597348 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 15 08:32:53.597348 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:53.597348 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 08:32:53.597348 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:53.597348 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:53.597348 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:53.597348 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:53.597348 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:53.598312 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:53.598312 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:53.598312 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:53.598312 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:53.598312 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:53.598312 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:53.598312 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:53.598312 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 08:32:53.598312 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 08:32:53.598312 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 08:32:53.598312 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 08:32:53.598312 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:53.598312 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:53.598312 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:53.598312 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:53.598312 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:53.598312 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:53.599191 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:53.599191 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:53.599191 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:53.599191 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:53.599191 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:53.599191 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:53.599191 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:53.599191 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:53.599191 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:53.599191 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:53.599191 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 08:32:53.599191 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 08:32:53.599191 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 15 08:32:53.599191 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:53.599191 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 337/670] 10.222.0.30 () {56 vars in 1132 bytes} [Sat Oct 15 08:32:53 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:53.604949 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe44d68-cf60-4a69-adec-eecc7d1d7308 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:53.709606 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe44d68-cf60-4a69-adec-eecc7d1d7308 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:53.749218 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed664ce0-784e-45f3-981e-dfa1277033fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:53.749861 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed664ce0-784e-45f3-981e-dfa1277033fe None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:53.750141 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed664ce0-784e-45f3-981e-dfa1277033fe None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:53.750417 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed664ce0-784e-45f3-981e-dfa1277033fe None admin] PATH_INFO: `/v3/services` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:53.751052 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed664ce0-784e-45f3-981e-dfa1277033fe None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:53.751362 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed664ce0-784e-45f3-981e-dfa1277033fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:53.752747 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed664ce0-784e-45f3-981e-dfa1277033fe None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:53.758601 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 334/671] 10.222.0.30 () {56 vars in 1126 bytes} [Sat Oct 15 08:32:53 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:53.766761 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5a1685-98fd-4b69-8a99-ba251854312c None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:53.842370 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20616e11-6015-4993-acfd-04bda02e89ea None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:53.843008 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20616e11-6015-4993-acfd-04bda02e89ea None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:53.843603 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20616e11-6015-4993-acfd-04bda02e89ea None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:53.845246 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 335/672] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:32:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 08:32:53.852174 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02886694-4613-4327-a8f0-025f1754c3c1 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:53.852749 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02886694-4613-4327-a8f0-025f1754c3c1 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:53.853354 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02886694-4613-4327-a8f0-025f1754c3c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:53.869512 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5a1685-98fd-4b69-8a99-ba251854312c None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:53.872602 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-02886694-4613-4327-a8f0-025f1754c3c1 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:32:53.873882 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-02886694-4613-4327-a8f0-025f1754c3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:32:53.912045 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e973f8-318b-4282-9086-12f30569e9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:53.912657 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e973f8-318b-4282-9086-12f30569e9f5 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:53.912939 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e973f8-318b-4282-9086-12f30569e9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:53.913214 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e973f8-318b-4282-9086-12f30569e9f5 None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:53.914455 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e973f8-318b-4282-9086-12f30569e9f5 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:53.915072 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e973f8-318b-4282-9086-12f30569e9f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:53.916439 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e973f8-318b-4282-9086-12f30569e9f5 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:53.919776 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:53.925854 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 338/673] 10.222.0.30 () {56 vars in 1126 bytes} [Sat Oct 15 08:32:53 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:53.927767 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e973f8-318b-4282-9086-12f30569e9f5 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:53.969377 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 336/674] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:32:53 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 08:32:54.000961 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:54.046664 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e973f8-318b-4282-9086-12f30569e9f5 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:54.082371 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49df9997-ab7f-434c-97a5-16258004fdb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:54.082994 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df9997-ab7f-434c-97a5-16258004fdb0 None admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:54.083256 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df9997-ab7f-434c-97a5-16258004fdb0 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:54.083532 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df9997-ab7f-434c-97a5-16258004fdb0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:54.084195 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49df9997-ab7f-434c-97a5-16258004fdb0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:54.085574 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49df9997-ab7f-434c-97a5-16258004fdb0 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:54.108201 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 339/675] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:32:53 2022] POST /identity/v3/endpoints => generated 347 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 08:32:54.245096 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02886694-4613-4327-a8f0-025f1754c3c1 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:54.353759 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02886694-4613-4327-a8f0-025f1754c3c1 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:54.406615 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb630d6e-fea2-4610-ad95-148d1fc2041b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:54.407653 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb630d6e-fea2-4610-ad95-148d1fc2041b None admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:54.408133 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb630d6e-fea2-4610-ad95-148d1fc2041b None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:54.408613 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb630d6e-fea2-4610-ad95-148d1fc2041b None admin] PATH_INFO: `/v3/services` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:54.409487 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb630d6e-fea2-4610-ad95-148d1fc2041b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:54.411191 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb630d6e-fea2-4610-ad95-148d1fc2041b None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:54.428874 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 337/676] 10.222.0.30 () {60 vars in 1141 bytes} [Sat Oct 15 08:32:54 2022] POST /identity/v3/services => generated 251 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:32:54.510005 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2886a-50c1-4cee-8272-489a1ddf570d None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:54.510301 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2886a-50c1-4cee-8272-489a1ddf570d None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:54.510475 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2886a-50c1-4cee-8272-489a1ddf570d None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:54.512248 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 340/677] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:32:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:32:54.518660 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e142f-fa6a-46f8-9f66-f35ac0d590cc None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:54.519290 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e142f-fa6a-46f8-9f66-f35ac0d590cc None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:54.519767 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e142f-fa6a-46f8-9f66-f35ac0d590cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:54.541360 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-8d3e142f-fa6a-46f8-9f66-f35ac0d590cc None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:32:54.543028 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-8d3e142f-fa6a-46f8-9f66-f35ac0d590cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:32:54.567955 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:54.630174 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 338/678] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:32:54 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 08:32:54.651457 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc058f11-0de2-4dda-a744-98a6c749a174 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:54.651753 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc058f11-0de2-4dda-a744-98a6c749a174 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:54.652019 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc058f11-0de2-4dda-a744-98a6c749a174 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:54.653494 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 341/679] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:32:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:32:54.659020 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e106e4e-e554-4357-9adf-31841fc5c8b6 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:54.659576 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e106e4e-e554-4357-9adf-31841fc5c8b6 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:54.660071 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e106e4e-e554-4357-9adf-31841fc5c8b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:54.681718 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-0e106e4e-e554-4357-9adf-31841fc5c8b6 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:32:54.682271 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-0e106e4e-e554-4357-9adf-31841fc5c8b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:32:54.753330 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 339/680] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:32:54 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 08:32:54.912138 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc058f11-0de2-4dda-a744-98a6c749a174 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:55.017612 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc058f11-0de2-4dda-a744-98a6c749a174 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:55.025597 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e106e4e-e554-4357-9adf-31841fc5c8b6 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:55.050162 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:55.065088 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5076d6-f7de-4f30-ba8b-077a14b09f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:55.065725 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5076d6-f7de-4f30-ba8b-077a14b09f53 None admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:55.066018 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5076d6-f7de-4f30-ba8b-077a14b09f53 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:55.066300 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5076d6-f7de-4f30-ba8b-077a14b09f53 None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:55.066974 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5076d6-f7de-4f30-ba8b-077a14b09f53 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:55.068922 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5076d6-f7de-4f30-ba8b-077a14b09f53 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:55.089089 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 342/681] 10.222.0.30 () {60 vars in 1141 bytes} [Sat Oct 15 08:32:54 2022] POST /identity/v3/services => generated 241 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:32:55.121113 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:55.144605 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e106e4e-e554-4357-9adf-31841fc5c8b6 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:55.190981 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5af874f-e4b1-4ce7-9418-4bd43e305f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:55.191618 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af874f-e4b1-4ce7-9418-4bd43e305f0b None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:55.191903 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af874f-e4b1-4ce7-9418-4bd43e305f0b None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:55.192178 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af874f-e4b1-4ce7-9418-4bd43e305f0b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:55.206073 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5af874f-e4b1-4ce7-9418-4bd43e305f0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:32:55.211393 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-a5af874f-e4b1-4ce7-9418-4bd43e305f0b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 15 08:32:55.211393 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:55.211393 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:55.211393 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:55.211393 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:55.211393 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:55.211393 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:55.211393 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:55.211393 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:55.211393 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:55.211393 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:55.211393 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:55.211393 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:32:55.211393 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:32:55.211393 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:32:55.211393 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:32:55.212459 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:32:55.212459 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:32:55.212459 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:32:55.212459 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:32:55.212459 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:55.212459 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:55.212459 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:55.212459 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:55.212459 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:55.212459 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:55.212459 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:55.212459 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:55.212459 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:55.212459 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:55.212459 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:55.212459 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:55.213334 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:55.213334 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:55.213334 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:55.213334 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:55.213334 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:32:55.213334 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:32:55.213334 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:32:55.213334 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:32:55.213334 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:32:55.213334 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:32:55.213334 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 15 08:32:55.213334 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:55.213334 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 340/682] 10.222.0.30 () {56 vars in 1120 bytes} [Sat Oct 15 08:32:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:55.219030 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5076d6-f7de-4f30-ba8b-077a14b09f53 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:55.333194 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5076d6-f7de-4f30-ba8b-077a14b09f53 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:55.368785 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c03da9b-509b-4683-af73-ef2b8b28a0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:55.369415 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c03da9b-509b-4683-af73-ef2b8b28a0c2 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:55.369721 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c03da9b-509b-4683-af73-ef2b8b28a0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:55.370007 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c03da9b-509b-4683-af73-ef2b8b28a0c2 None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:55.370695 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c03da9b-509b-4683-af73-ef2b8b28a0c2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:55.371002 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c03da9b-509b-4683-af73-ef2b8b28a0c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:55.372407 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c03da9b-509b-4683-af73-ef2b8b28a0c2 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:55.382038 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 343/683] 10.222.0.30 () {56 vars in 1115 bytes} [Sat Oct 15 08:32:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:55.390811 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5af874f-e4b1-4ce7-9418-4bd43e305f0b None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:55.501412 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5af874f-e4b1-4ce7-9418-4bd43e305f0b None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:55.538061 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e40c0aa-ae12-4c2b-9ff8-081793f889ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:55.539129 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e40c0aa-ae12-4c2b-9ff8-081793f889ee None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:55.539628 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e40c0aa-ae12-4c2b-9ff8-081793f889ee None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:55.540112 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e40c0aa-ae12-4c2b-9ff8-081793f889ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:55.547379 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:55.552193 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e40c0aa-ae12-4c2b-9ff8-081793f889ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:55.554043 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e40c0aa-ae12-4c2b-9ff8-081793f889ee None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:55.570264 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-3e40c0aa-ae12-4c2b-9ff8-081793f889ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:55.570264 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:55.570264 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:55.570264 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:55.570264 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:55.570264 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:55.570264 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:55.570264 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:55.570264 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:55.570264 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:55.570264 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:55.570264 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:55.570264 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 08:32:55.570264 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 08:32:55.570264 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 08:32:55.570264 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 08:32:55.571240 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:55.571240 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:55.571240 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:55.571240 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:55.571240 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:55.571240 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:55.571240 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:55.571240 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:55.571240 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:55.571240 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:55.571240 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:55.571240 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:55.571240 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:55.571240 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:55.571240 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:55.571240 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:55.571240 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 08:32:55.572137 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 08:32:55.572137 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:55.572137 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:55.574218 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 341/684] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:32:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 08:32:55.580138 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c03da9b-509b-4683-af73-ef2b8b28a0c2 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:55.618437 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:55.692757 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c03da9b-509b-4683-af73-ef2b8b28a0c2 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:55.727864 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794205bf-8ef4-4312-989b-1d799d27196d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:55.728814 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794205bf-8ef4-4312-989b-1d799d27196d None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:55.729319 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794205bf-8ef4-4312-989b-1d799d27196d None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:55.729817 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794205bf-8ef4-4312-989b-1d799d27196d None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:55.730649 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794205bf-8ef4-4312-989b-1d799d27196d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:55.731161 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794205bf-8ef4-4312-989b-1d799d27196d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:55.732714 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794205bf-8ef4-4312-989b-1d799d27196d None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:55.745284 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 344/685] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 08:32:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:55.751076 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e40c0aa-ae12-4c2b-9ff8-081793f889ee None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:55.858958 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e40c0aa-ae12-4c2b-9ff8-081793f889ee None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:55.899355 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf691a2-f6a8-446c-8bf6-0d12ffde6652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:55.900327 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf691a2-f6a8-446c-8bf6-0d12ffde6652 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:55.900838 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf691a2-f6a8-446c-8bf6-0d12ffde6652 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:55.901340 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf691a2-f6a8-446c-8bf6-0d12ffde6652 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:55.909277 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf691a2-f6a8-446c-8bf6-0d12ffde6652 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:55.910990 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf691a2-f6a8-446c-8bf6-0d12ffde6652 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:55.923102 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-caf691a2-f6a8-446c-8bf6-0d12ffde6652 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 15 08:32:55.923102 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:55.923102 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:55.923102 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:55.923102 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:55.923102 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:55.923102 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:55.923102 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:55.923102 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:55.923102 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:55.923102 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:55.923102 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:55.923102 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 08:32:55.923102 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 08:32:55.923102 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 08:32:55.923102 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 08:32:55.924037 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:55.924037 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:55.924037 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 08:32:55.924037 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 08:32:55.924037 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 08:32:55.924037 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 08:32:55.924037 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:55.924037 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:55.924037 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:55.924037 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:55.924037 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:55.924037 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:55.924037 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:55.924037 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:55.924037 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:55.924037 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:55.924037 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:55.924037 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:55.924989 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:55.924989 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:55.924989 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 08:32:55.924989 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 08:32:55.924989 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 08:32:55.924989 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 08:32:55.924989 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 08:32:55.924989 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 08:32:55.924989 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 15 08:32:55.924989 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:55.927465 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 342/686] 10.222.0.30 () {56 vars in 1117 bytes} [Sat Oct 15 08:32:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:55.933487 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794205bf-8ef4-4312-989b-1d799d27196d None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:56.031148 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:56.037628 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794205bf-8ef4-4312-989b-1d799d27196d None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:56.075494 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e1427-b07c-4f35-b3e7-373f0d5522cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:56.076108 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e1427-b07c-4f35-b3e7-373f0d5522cd None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:56.076391 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e1427-b07c-4f35-b3e7-373f0d5522cd None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:56.076666 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e1427-b07c-4f35-b3e7-373f0d5522cd None admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:56.077393 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295e1427-b07c-4f35-b3e7-373f0d5522cd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:56.077716 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295e1427-b07c-4f35-b3e7-373f0d5522cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:56.079145 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295e1427-b07c-4f35-b3e7-373f0d5522cd None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:56.097080 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 345/687] 10.222.0.30 () {56 vars in 1149 bytes} [Sat Oct 15 08:32:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:56.106773 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf691a2-f6a8-446c-8bf6-0d12ffde6652 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:56.160570 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:56.212848 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf691a2-f6a8-446c-8bf6-0d12ffde6652 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:56.247831 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed99ad9d-9e91-41ec-a8bc-212d17502fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:56.248490 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed99ad9d-9e91-41ec-a8bc-212d17502fbe None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:56.248777 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed99ad9d-9e91-41ec-a8bc-212d17502fbe None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:56.249053 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed99ad9d-9e91-41ec-a8bc-212d17502fbe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:56.320659 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed99ad9d-9e91-41ec-a8bc-212d17502fbe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:56.322282 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed99ad9d-9e91-41ec-a8bc-212d17502fbe None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:56.375273 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:56.436985 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 343/688] 10.222.0.30 () {58 vars in 1282 bytes} [Sat Oct 15 08:32:56 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 15 08:32:56.442280 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e1427-b07c-4f35-b3e7-373f0d5522cd None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:56.456324 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd2b800-ce5e-4866-84c4-7032221f7974 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:56.456705 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd2b800-ce5e-4866-84c4-7032221f7974 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:56.456988 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd2b800-ce5e-4866-84c4-7032221f7974 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:56.459286 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 344/689] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:32:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 08:32:56.465254 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716af3e1-78fe-46fa-9f61-eac12bd75daa None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:56.465539 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716af3e1-78fe-46fa-9f61-eac12bd75daa None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:56.465856 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716af3e1-78fe-46fa-9f61-eac12bd75daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:56.489056 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-716af3e1-78fe-46fa-9f61-eac12bd75daa None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:32:56.490274 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-716af3e1-78fe-46fa-9f61-eac12bd75daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:32:56.564316 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e1427-b07c-4f35-b3e7-373f0d5522cd None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:56.573177 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 345/690] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:32:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 08:32:56.613079 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c79396a-0867-4e09-be62-d8f9eec95f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:56.613753 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c79396a-0867-4e09-be62-d8f9eec95f9a None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:56.614038 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c79396a-0867-4e09-be62-d8f9eec95f9a None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:56.614312 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c79396a-0867-4e09-be62-d8f9eec95f9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:56.626125 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c79396a-0867-4e09-be62-d8f9eec95f9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:56.627625 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c79396a-0867-4e09-be62-d8f9eec95f9a None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:56.643361 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-6c79396a-0867-4e09-be62-d8f9eec95f9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:56.643361 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:56.643361 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:56.643361 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:56.643361 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:56.643361 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:56.643361 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:56.643361 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:56.643361 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:56.643361 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:56.643361 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:56.643361 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:56.643361 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 08:32:56.643361 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 08:32:56.643361 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 08:32:56.643361 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 08:32:56.644289 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:56.644289 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:56.644289 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:56.644289 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:56.644289 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:56.644289 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:56.644289 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:56.644289 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:56.644289 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:56.644289 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:56.644289 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:56.644289 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:56.644289 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:56.644289 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:56.644289 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:56.644289 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:56.644289 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 08:32:56.645174 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 08:32:56.645174 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:32:56.645174 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:56.645174 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 346/691] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:32:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:56.650534 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716af3e1-78fe-46fa-9f61-eac12bd75daa None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:56.705127 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c790c-1cf2-4fa3-b5f0-d51af4e636a8 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:56.705412 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c790c-1cf2-4fa3-b5f0-d51af4e636a8 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:56.705605 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c790c-1cf2-4fa3-b5f0-d51af4e636a8 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:56.707253 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 347/692] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:32:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:32:56.714889 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb8720d-acca-4597-ac11-38a9b0d3cba1 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:56.715276 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb8720d-acca-4597-ac11-38a9b0d3cba1 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:56.716006 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb8720d-acca-4597-ac11-38a9b0d3cba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:56.725245 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:56.741393 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-1cb8720d-acca-4597-ac11-38a9b0d3cba1 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:32:56.741555 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:56.743573 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-1cb8720d-acca-4597-ac11-38a9b0d3cba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:32:56.763152 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716af3e1-78fe-46fa-9f61-eac12bd75daa None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:56.799028 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f32368-8470-408f-b2b4-fb207ec73ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:56.799713 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f32368-8470-408f-b2b4-fb207ec73ec8 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:56.799996 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f32368-8470-408f-b2b4-fb207ec73ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:56.800277 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f32368-8470-408f-b2b4-fb207ec73ec8 None admin] PATH_INFO: `/v3/domains` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:56.800958 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f32368-8470-408f-b2b4-fb207ec73ec8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:56.801265 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f32368-8470-408f-b2b4-fb207ec73ec8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:56.803311 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f32368-8470-408f-b2b4-fb207ec73ec8 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:56.822262 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 346/693] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 08:32:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:56.824111 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f32368-8470-408f-b2b4-fb207ec73ec8 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:56.834252 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 348/694] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:32:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 08:32:56.858457 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb8720d-acca-4597-ac11-38a9b0d3cba1 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:56.935277 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f32368-8470-408f-b2b4-fb207ec73ec8 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:56.978496 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22248a18-abc1-430c-a966-4ffc7c3ffb17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:56.979044 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22248a18-abc1-430c-a966-4ffc7c3ffb17 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:56.979320 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22248a18-abc1-430c-a966-4ffc7c3ffb17 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:56.979605 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22248a18-abc1-430c-a966-4ffc7c3ffb17 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:56.985450 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb8720d-acca-4597-ac11-38a9b0d3cba1 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:56.986677 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22248a18-abc1-430c-a966-4ffc7c3ffb17 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:56.988162 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22248a18-abc1-430c-a966-4ffc7c3ffb17 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:56.991013 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:57.001000 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-22248a18-abc1-430c-a966-4ffc7c3ffb17 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 08:32:57.001000 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:57.001000 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:57.001000 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:57.001000 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:57.001000 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:57.001000 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:57.001000 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:57.001000 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:57.001000 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:57.001000 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:57.001000 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:57.001000 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 08:32:57.001000 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 08:32:57.001000 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 08:32:57.001985 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 08:32:57.001985 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:57.001985 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:57.001985 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:57.001985 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:57.001985 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:57.001985 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:57.001985 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:57.001985 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:57.001985 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:57.001985 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:57.001985 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:57.001985 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:57.001985 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:57.001985 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:57.001985 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:57.001985 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:57.002980 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 08:32:57.002980 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 08:32:57.002980 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 08:32:57.002980 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 08:32:57.002980 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 08:32:57.002980 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 08:32:57.002980 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 08:32:57.002980 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:57.008345 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 347/695] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 08:32:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:57.009209 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22248a18-abc1-430c-a966-4ffc7c3ffb17 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:57.034183 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93d4b4b-ff8c-4dad-ba40-f47b1fef3b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:57.035897 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93d4b4b-ff8c-4dad-ba40-f47b1fef3b7c None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:57.036697 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93d4b4b-ff8c-4dad-ba40-f47b1fef3b7c None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:57.037179 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93d4b4b-ff8c-4dad-ba40-f47b1fef3b7c None admin] PATH_INFO: `/v3/services/compute` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:57.038245 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93d4b4b-ff8c-4dad-ba40-f47b1fef3b7c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:57.040621 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93d4b4b-ff8c-4dad-ba40-f47b1fef3b7c None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:57.043625 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:57.055795 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-b93d4b4b-ff8c-4dad-ba40-f47b1fef3b7c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 15 08:32:57.055795 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:57.055795 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 08:32:57.055795 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 08:32:57.055795 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 08:32:57.055795 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 08:32:57.055795 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 08:32:57.055795 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 08:32:57.055795 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 15 08:32:57.055795 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:57.055795 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 08:32:57.055795 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:57.055795 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:57.055795 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:57.055795 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:57.055795 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:57.055795 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:57.056844 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:57.056844 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:57.056844 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:57.056844 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:57.056844 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:57.056844 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:57.056844 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 08:32:57.056844 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 08:32:57.056844 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 08:32:57.056844 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 08:32:57.056844 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:57.056844 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:57.056844 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:57.056844 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:57.056844 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:57.056844 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:57.057759 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:57.057759 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:57.057759 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:57.057759 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:57.057759 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:57.057759 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:57.057759 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:57.057759 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:57.057759 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:57.057759 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:57.057759 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 08:32:57.057759 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 08:32:57.057759 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 15 08:32:57.057759 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:57.061846 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 349/696] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 08:32:56 2022] GET /identity/v3/services/compute => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:57.064731 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93d4b4b-ff8c-4dad-ba40-f47b1fef3b7c None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:57.124187 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22248a18-abc1-430c-a966-4ffc7c3ffb17 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:57.166885 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749b2c9e-f913-43e3-879b-071ca4da8e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:57.167497 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b2c9e-f913-43e3-879b-071ca4da8e0c None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:57.167776 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b2c9e-f913-43e3-879b-071ca4da8e0c None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:57.168047 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b2c9e-f913-43e3-879b-071ca4da8e0c None admin] PATH_INFO: `/v3/projects` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:57.168751 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749b2c9e-f913-43e3-879b-071ca4da8e0c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:57.169034 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749b2c9e-f913-43e3-879b-071ca4da8e0c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:57.170491 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749b2c9e-f913-43e3-879b-071ca4da8e0c None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:57.187223 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 348/697] 10.222.0.30 () {56 vars in 1148 bytes} [Sat Oct 15 08:32:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:57.187223 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749b2c9e-f913-43e3-879b-071ca4da8e0c None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:57.195013 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93d4b4b-ff8c-4dad-ba40-f47b1fef3b7c None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:57.234696 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147048d7-f015-4b4d-8a23-4a233992e778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:57.235917 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147048d7-f015-4b4d-8a23-4a233992e778 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:57.236353 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147048d7-f015-4b4d-8a23-4a233992e778 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:57.236795 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147048d7-f015-4b4d-8a23-4a233992e778 None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:57.237599 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147048d7-f015-4b4d-8a23-4a233992e778 None admin] RBAC: Adding query filter params (name=compute) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:57.238511 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147048d7-f015-4b4d-8a23-4a233992e778 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:57.240428 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147048d7-f015-4b4d-8a23-4a233992e778 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:57.244615 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:57.247318 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 350/698] 10.222.0.30 () {56 vars in 1112 bytes} [Sat Oct 15 08:32:57 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:57.248962 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147048d7-f015-4b4d-8a23-4a233992e778 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:57.290147 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749b2c9e-f913-43e3-879b-071ca4da8e0c None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:57.320772 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:57.325233 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556dece2-ab54-427e-bc72-dbb97897727f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:57.325858 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556dece2-ab54-427e-bc72-dbb97897727f None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:57.326138 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556dece2-ab54-427e-bc72-dbb97897727f None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:57.326429 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556dece2-ab54-427e-bc72-dbb97897727f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:57.326974 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556dece2-ab54-427e-bc72-dbb97897727f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:57.328341 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556dece2-ab54-427e-bc72-dbb97897727f None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:57.337384 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-556dece2-ab54-427e-bc72-dbb97897727f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 15 08:32:57.337384 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:57.337384 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 08:32:57.337384 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 08:32:57.337384 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 08:32:57.337384 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 08:32:57.337384 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 08:32:57.337384 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 08:32:57.337384 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 15 08:32:57.337384 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:57.337384 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 08:32:57.337384 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:57.337384 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:57.337384 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:57.337384 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:57.337384 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:57.337384 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:57.338696 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:57.338696 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:57.338696 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:57.338696 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:57.338696 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:57.338696 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:57.338696 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 08:32:57.338696 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 08:32:57.338696 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 08:32:57.338696 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 08:32:57.338696 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:57.338696 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:57.338696 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:57.338696 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:57.338696 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:57.338696 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:57.339619 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:57.339619 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:57.339619 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:57.339619 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:57.339619 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:57.339619 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:57.339619 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:57.339619 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:57.339619 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:57.339619 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:57.339619 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 08:32:57.339619 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 08:32:57.339619 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 15 08:32:57.339619 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:32:57.339619 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 349/699] 10.222.0.30 () {56 vars in 1114 bytes} [Sat Oct 15 08:32:57 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:32:57.341573 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556dece2-ab54-427e-bc72-dbb97897727f None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:57.373289 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147048d7-f015-4b4d-8a23-4a233992e778 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:57.417138 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fe1d21-2454-4dbc-be48-c950f88db07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:57.418566 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe1d21-2454-4dbc-be48-c950f88db07d None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:57.418862 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe1d21-2454-4dbc-be48-c950f88db07d None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:57.419139 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe1d21-2454-4dbc-be48-c950f88db07d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:57.419923 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fe1d21-2454-4dbc-be48-c950f88db07d None admin] RBAC: Adding query filter params (role.id=3e2674a06b1e46739ff0be6a5e20003e, scope.project.id=5dfceb59b3794439a4586b3cbe408f0e, user.id=c5b2a0e623c348ad94d4820080677be6) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:57.420233 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fe1d21-2454-4dbc-be48-c950f88db07d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:57.421753 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fe1d21-2454-4dbc-be48-c950f88db07d None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:57.429368 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 351/700] 10.222.0.30 () {56 vars in 1374 bytes} [Sat Oct 15 08:32:57 2022] GET /identity/v3/role_assignments?user.id=c5b2a0e623c348ad94d4820080677be6&scope.project.id=5dfceb59b3794439a4586b3cbe408f0e&role.id=3e2674a06b1e46739ff0be6a5e20003e => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:57.434735 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fe1d21-2454-4dbc-be48-c950f88db07d None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:57.462548 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556dece2-ab54-427e-bc72-dbb97897727f None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:57.489508 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:57.501843 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de9a5f3-3ea3-4fca-be00-31c2887b4ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:57.502382 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9a5f3-3ea3-4fca-be00-31c2887b4ef8 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:57.502666 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9a5f3-3ea3-4fca-be00-31c2887b4ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:57.502977 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9a5f3-3ea3-4fca-be00-31c2887b4ef8 None admin] PATH_INFO: `/v3/services` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:57.503584 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de9a5f3-3ea3-4fca-be00-31c2887b4ef8 None admin] RBAC: Adding query filter params (type=compute) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:57.503892 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de9a5f3-3ea3-4fca-be00-31c2887b4ef8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:57.505301 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de9a5f3-3ea3-4fca-be00-31c2887b4ef8 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:57.517191 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 350/701] 10.222.0.30 () {56 vars in 1112 bytes} [Sat Oct 15 08:32:57 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:57.517191 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d67ccb3-a9f7-44e7-bfdc-3737cc800e4a None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:57.517191 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d67ccb3-a9f7-44e7-bfdc-3737cc800e4a None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:57.517191 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d67ccb3-a9f7-44e7-bfdc-3737cc800e4a None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:57.517191 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 351/702] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:32:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:32:57.520332 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182478f1-941d-4a6c-ba79-f414afbcae5a None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:57.520646 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182478f1-941d-4a6c-ba79-f414afbcae5a None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:57.521009 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182478f1-941d-4a6c-ba79-f414afbcae5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:57.541960 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-182478f1-941d-4a6c-ba79-f414afbcae5a None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:32:57.543219 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-182478f1-941d-4a6c-ba79-f414afbcae5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:32:57.560607 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fe1d21-2454-4dbc-be48-c950f88db07d None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:57.596901 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:57.609624 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c768241-3fed-427d-ac53-464571026723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:57.610306 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c768241-3fed-427d-ac53-464571026723 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:57.610597 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c768241-3fed-427d-ac53-464571026723 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:57.611045 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c768241-3fed-427d-ac53-464571026723 None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:57.611711 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c768241-3fed-427d-ac53-464571026723 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:57.612071 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c768241-3fed-427d-ac53-464571026723 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:57.613501 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c768241-3fed-427d-ac53-464571026723 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:57.619471 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 352/703] 10.222.0.30 () {56 vars in 1114 bytes} [Sat Oct 15 08:32:57 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:57.625396 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c768241-3fed-427d-ac53-464571026723 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:57.637030 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 352/704] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:32:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 08:32:57.741130 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c768241-3fed-427d-ac53-464571026723 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:57.776762 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5a4ac1-147c-422c-be6f-2884be09ba55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:57.777794 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a4ac1-147c-422c-be6f-2884be09ba55 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:57.778334 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a4ac1-147c-422c-be6f-2884be09ba55 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:57.778812 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a4ac1-147c-422c-be6f-2884be09ba55 None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:57.779653 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5a4ac1-147c-422c-be6f-2884be09ba55 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:57.780157 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5a4ac1-147c-422c-be6f-2884be09ba55 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:57.781801 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5a4ac1-147c-422c-be6f-2884be09ba55 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:57.787325 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 353/705] 10.222.0.30 () {56 vars in 1114 bytes} [Sat Oct 15 08:32:57 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 08:32:57.793422 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182478f1-941d-4a6c-ba79-f414afbcae5a None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:57.902178 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182478f1-941d-4a6c-ba79-f414afbcae5a None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:57.904369 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5a4ac1-147c-422c-be6f-2884be09ba55 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:57.923000 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:57.947498 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e3c7d1-351f-4dbc-96bb-b1dfd6be93a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:57.948154 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e3c7d1-351f-4dbc-96bb-b1dfd6be93a5 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:57.948435 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e3c7d1-351f-4dbc-96bb-b1dfd6be93a5 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:57.948761 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e3c7d1-351f-4dbc-96bb-b1dfd6be93a5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:57.949464 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e3c7d1-351f-4dbc-96bb-b1dfd6be93a5 None admin] RBAC: Adding query filter params (interface=public, service_id=1c31719e1e9d43158c5d370e9f561bc4, region_id=RegionOne) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:57.949980 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e3c7d1-351f-4dbc-96bb-b1dfd6be93a5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:57.951675 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e3c7d1-351f-4dbc-96bb-b1dfd6be93a5 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:57.959722 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 353/706] 10.222.0.30 () {56 vars in 1251 bytes} [Sat Oct 15 08:32:57 2022] GET /identity/v3/endpoints?service_id=1c31719e1e9d43158c5d370e9f561bc4&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:57.964926 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e3c7d1-351f-4dbc-96bb-b1dfd6be93a5 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:57.974733 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:58.026520 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5a4ac1-147c-422c-be6f-2884be09ba55 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:58.063933 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c22543-0b80-45f4-ad30-3130ef9f3bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:58.064570 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c22543-0b80-45f4-ad30-3130ef9f3bea None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:58.064856 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c22543-0b80-45f4-ad30-3130ef9f3bea None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:58.065189 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c22543-0b80-45f4-ad30-3130ef9f3bea None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:58.065823 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c22543-0b80-45f4-ad30-3130ef9f3bea None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:58.067306 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c22543-0b80-45f4-ad30-3130ef9f3bea None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:58.077038 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-f2c22543-0b80-45f4-ad30-3130ef9f3bea None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 15 08:32:58.077038 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:58.077038 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 08:32:58.077038 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 08:32:58.077038 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 08:32:58.077038 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 08:32:58.077038 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 08:32:58.077038 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 08:32:58.077038 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 15 08:32:58.077038 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:58.077038 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 08:32:58.077038 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:58.077038 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:32:58.077038 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:32:58.077038 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:32:58.077038 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:32:58.077038 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:32:58.078105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:32:58.078105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:32:58.078105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:32:58.078105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:32:58.078105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:32:58.078105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:32:58.078105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 08:32:58.078105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 08:32:58.078105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 08:32:58.078105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 08:32:58.078105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:32:58.078105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:32:58.078105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:32:58.078105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:32:58.078105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:32:58.078105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:32:58.078977 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:32:58.078977 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:32:58.078977 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:32:58.078977 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:32:58.078977 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:32:58.078977 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:32:58.078977 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:32:58.078977 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:32:58.078977 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:32:58.078977 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:32:58.078977 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 08:32:58.078977 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 08:32:58.078977 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 15 08:32:58.078977 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:32:58.078977 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 354/707] 10.222.0.30 () {56 vars in 1126 bytes} [Sat Oct 15 08:32:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 08:32:58.085452 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c22543-0b80-45f4-ad30-3130ef9f3bea None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:58.106565 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e3c7d1-351f-4dbc-96bb-b1dfd6be93a5 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:58.154472 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0054d6dd-14e8-4607-98cf-c81c98594501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:58.156077 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0054d6dd-14e8-4607-98cf-c81c98594501 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:58.156641 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0054d6dd-14e8-4607-98cf-c81c98594501 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:58.157326 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0054d6dd-14e8-4607-98cf-c81c98594501 None admin] PATH_INFO: `/v3/services` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:58.158365 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0054d6dd-14e8-4607-98cf-c81c98594501 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:58.160599 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0054d6dd-14e8-4607-98cf-c81c98594501 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:58.169554 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 354/708] 10.222.0.30 () {56 vars in 1087 bytes} [Sat Oct 15 08:32:57 2022] GET /identity/v3/services => generated 1497 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 08:32:58.206114 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c22543-0b80-45f4-ad30-3130ef9f3bea None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:58.227889 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:58.247705 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d083adce-5ed9-44d0-b9a6-f3acb367e81b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:58.248341 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d083adce-5ed9-44d0-b9a6-f3acb367e81b None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:58.248624 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d083adce-5ed9-44d0-b9a6-f3acb367e81b None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:58.248901 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d083adce-5ed9-44d0-b9a6-f3acb367e81b None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:58.249623 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d083adce-5ed9-44d0-b9a6-f3acb367e81b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:58.250024 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d083adce-5ed9-44d0-b9a6-f3acb367e81b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:58.251743 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d083adce-5ed9-44d0-b9a6-f3acb367e81b None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:58.258330 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 355/709] 10.222.0.30 () {56 vars in 1122 bytes} [Sat Oct 15 08:32:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:58.264077 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0054d6dd-14e8-4607-98cf-c81c98594501 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:58.318141 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:58.380273 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0054d6dd-14e8-4607-98cf-c81c98594501 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:58.421235 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf988097-6e18-48c7-bd1c-a7f02c10c7f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:58.421896 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf988097-6e18-48c7-bd1c-a7f02c10c7f0 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:58.422185 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf988097-6e18-48c7-bd1c-a7f02c10c7f0 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:58.422461 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf988097-6e18-48c7-bd1c-a7f02c10c7f0 None admin] PATH_INFO: `/v3/services` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:58.423108 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf988097-6e18-48c7-bd1c-a7f02c10c7f0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:58.423418 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf988097-6e18-48c7-bd1c-a7f02c10c7f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:58.424847 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf988097-6e18-48c7-bd1c-a7f02c10c7f0 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:58.431121 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 355/710] 10.222.0.30 () {56 vars in 1122 bytes} [Sat Oct 15 08:32:58 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:58.437322 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d083adce-5ed9-44d0-b9a6-f3acb367e81b None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:58.546267 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d083adce-5ed9-44d0-b9a6-f3acb367e81b None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:58.581217 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6714d68-3cba-4b72-b26e-89c42aaf7161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:58.581776 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6714d68-3cba-4b72-b26e-89c42aaf7161 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:58.582061 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6714d68-3cba-4b72-b26e-89c42aaf7161 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:58.582336 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6714d68-3cba-4b72-b26e-89c42aaf7161 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:58.583035 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6714d68-3cba-4b72-b26e-89c42aaf7161 None admin] RBAC: Adding query filter params (interface=public, service_id=fc0069003c8e49f892e5afb67ea77d16, region_id=RegionOne) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:32:58.583343 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6714d68-3cba-4b72-b26e-89c42aaf7161 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:58.584710 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6714d68-3cba-4b72-b26e-89c42aaf7161 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:58.592576 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 356/711] 10.222.0.30 () {56 vars in 1251 bytes} [Sat Oct 15 08:32:58 2022] GET /identity/v3/endpoints?service_id=fc0069003c8e49f892e5afb67ea77d16&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:32:58.598189 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf988097-6e18-48c7-bd1c-a7f02c10c7f0 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:32:58.701761 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf988097-6e18-48c7-bd1c-a7f02c10c7f0 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:32:58.713550 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:58.738373 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224a2f2-7eb9-4482-a6e6-8211419fd091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:32:58.738984 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224a2f2-7eb9-4482-a6e6-8211419fd091 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:58.739269 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224a2f2-7eb9-4482-a6e6-8211419fd091 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:58.739549 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224a2f2-7eb9-4482-a6e6-8211419fd091 None admin] PATH_INFO: `/v3/services` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:58.740152 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9224a2f2-7eb9-4482-a6e6-8211419fd091 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:32:58.741525 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9224a2f2-7eb9-4482-a6e6-8211419fd091 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:32:58.748198 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 356/712] 10.222.0.30 () {56 vars in 1087 bytes} [Sat Oct 15 08:32:58 2022] GET /identity/v3/services => generated 1497 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:32:59.758800 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cea188e-c171-455a-945d-417364c7499e None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:59.759314 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cea188e-c171-455a-945d-417364c7499e None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:59.759393 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cea188e-c171-455a-945d-417364c7499e None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:59.760945 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 357/713] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:32:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:32:59.766692 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee5f783-faba-47a5-9a8a-d987e3a1e5f4 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:59.767348 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee5f783-faba-47a5-9a8a-d987e3a1e5f4 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:59.767536 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee5f783-faba-47a5-9a8a-d987e3a1e5f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:59.769317 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f034f0-4943-4222-83e2-f5ae07b63929 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:59.769606 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f034f0-4943-4222-83e2-f5ae07b63929 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:59.769930 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f034f0-4943-4222-83e2-f5ae07b63929 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:59.771524 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 358/714] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:32:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 08:32:59.776261 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d41f1f-f791-4f5d-a674-f940f0ef2cb8 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:32:59.776556 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d41f1f-f791-4f5d-a674-f940f0ef2cb8 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:32:59.776836 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d41f1f-f791-4f5d-a674-f940f0ef2cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:32:59.788225 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-aee5f783-faba-47a5-9a8a-d987e3a1e5f4 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:32:59.790249 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-aee5f783-faba-47a5-9a8a-d987e3a1e5f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:32:59.795901 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-e1d41f1f-f791-4f5d-a674-f940f0ef2cb8 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:32:59.797070 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-e1d41f1f-f791-4f5d-a674-f940f0ef2cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:32:59.842242 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:32:59.877783 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 359/715] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:32:59 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 08:32:59.885208 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 357/716] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:32:59 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 08:33:00.141838 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d41f1f-f791-4f5d-a674-f940f0ef2cb8 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:00.158105 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee5f783-faba-47a5-9a8a-d987e3a1e5f4 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:00.249871 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d41f1f-f791-4f5d-a674-f940f0ef2cb8 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:00.263755 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee5f783-faba-47a5-9a8a-d987e3a1e5f4 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:00.266349 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:00.289895 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1102da-399a-4a2c-83cf-dd91ecfc13e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:00.290850 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1102da-399a-4a2c-83cf-dd91ecfc13e0 None admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:00.291285 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1102da-399a-4a2c-83cf-dd91ecfc13e0 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:00.291736 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1102da-399a-4a2c-83cf-dd91ecfc13e0 None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:00.292534 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1102da-399a-4a2c-83cf-dd91ecfc13e0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:00.294279 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1102da-399a-4a2c-83cf-dd91ecfc13e0 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:00.302360 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1782c9f7-1a50-4ac2-8ded-d5709905dab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:00.303066 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1782c9f7-1a50-4ac2-8ded-d5709905dab5 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:00.303367 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1782c9f7-1a50-4ac2-8ded-d5709905dab5 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:00.303630 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1782c9f7-1a50-4ac2-8ded-d5709905dab5 None admin] PATH_INFO: `/v3/services/image` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:00.304222 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1782c9f7-1a50-4ac2-8ded-d5709905dab5 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:00.306520 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1782c9f7-1a50-4ac2-8ded-d5709905dab5 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:00.312821 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 360/717] 10.222.0.30 () {60 vars in 1141 bytes} [Sat Oct 15 08:33:00 2022] POST /identity/v3/services => generated 242 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:33:00.318596 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-1782c9f7-1a50-4ac2-8ded-d5709905dab5 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 15 08:33:00.318596 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:33:00.318596 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 08:33:00.318596 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 08:33:00.318596 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 08:33:00.318596 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 08:33:00.318596 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 08:33:00.318596 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 08:33:00.318596 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 15 08:33:00.318596 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:33:00.318596 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 08:33:00.318596 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:33:00.318596 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:33:00.318596 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:33:00.318596 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:33:00.318596 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:33:00.318596 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:33:00.319845 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:33:00.319845 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:33:00.319845 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:33:00.319845 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:33:00.319845 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:33:00.319845 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:33:00.319845 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 08:33:00.319845 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 08:33:00.319845 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 08:33:00.319845 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 08:33:00.319845 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:33:00.319845 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:33:00.319845 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:33:00.319845 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:33:00.319845 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:33:00.319845 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:33:00.320987 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:33:00.320987 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:33:00.320987 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:33:00.320987 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:33:00.320987 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:33:00.320987 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:33:00.320987 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:33:00.320987 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:33:00.320987 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:33:00.320987 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:33:00.320987 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 08:33:00.320987 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 08:33:00.320987 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 15 08:33:00.320987 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:33:00.320987 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 358/718] 10.222.0.30 () {56 vars in 1105 bytes} [Sat Oct 15 08:33:00 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 08:33:00.323593 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1102da-399a-4a2c-83cf-dd91ecfc13e0 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:00.368979 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:00.452364 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1102da-399a-4a2c-83cf-dd91ecfc13e0 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:00.488010 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd30f8a-47b6-4339-9c42-96f847275d09 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:00.488751 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd30f8a-47b6-4339-9c42-96f847275d09 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:00.489262 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd30f8a-47b6-4339-9c42-96f847275d09 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:00.491372 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 359/719] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:33:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 08:33:00.494885 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102947ac-bcad-4723-a7e1-721625cad6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:00.494885 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102947ac-bcad-4723-a7e1-721625cad6e3 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:00.494885 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102947ac-bcad-4723-a7e1-721625cad6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:00.494885 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102947ac-bcad-4723-a7e1-721625cad6e3 None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:00.495282 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102947ac-bcad-4723-a7e1-721625cad6e3 None admin] RBAC: Adding query filter params (name=image) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:33:00.495282 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102947ac-bcad-4723-a7e1-721625cad6e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:00.496650 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102947ac-bcad-4723-a7e1-721625cad6e3 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:00.498305 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf83ceb6-2a6b-4690-96da-d1960690c7db None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:00.498826 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf83ceb6-2a6b-4690-96da-d1960690c7db None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:00.499363 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf83ceb6-2a6b-4690-96da-d1960690c7db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:00.501954 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 361/720] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:33:00 2022] GET /identity/v3/services?name=image => generated 122 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 08:33:00.510736 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102947ac-bcad-4723-a7e1-721625cad6e3 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:00.526126 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-cf83ceb6-2a6b-4690-96da-d1960690c7db None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:33:00.528267 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-cf83ceb6-2a6b-4690-96da-d1960690c7db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:33:00.613880 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 360/721] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:33:00 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 08:33:00.639691 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:00.646214 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102947ac-bcad-4723-a7e1-721625cad6e3 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:00.681833 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78feacda-d714-4a0c-91a0-758745f8a1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:00.682419 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78feacda-d714-4a0c-91a0-758745f8a1bb None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:00.682696 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78feacda-d714-4a0c-91a0-758745f8a1bb None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:00.682968 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78feacda-d714-4a0c-91a0-758745f8a1bb None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:00.683645 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78feacda-d714-4a0c-91a0-758745f8a1bb None admin] RBAC: Adding query filter params (type=image) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:33:00.683950 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78feacda-d714-4a0c-91a0-758745f8a1bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:00.685301 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78feacda-d714-4a0c-91a0-758745f8a1bb None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:00.690361 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 362/722] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:33:00 2022] GET /identity/v3/services?type=image => generated 122 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:33:00.886650 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf83ceb6-2a6b-4690-96da-d1960690c7db None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:00.941528 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3429e507-b0b9-401e-bd40-06b8e5e3015d None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:00.942047 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3429e507-b0b9-401e-bd40-06b8e5e3015d None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:00.942047 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3429e507-b0b9-401e-bd40-06b8e5e3015d None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:00.943626 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 363/723] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:33:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 08:33:00.949114 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06794b57-5606-406d-965e-e6602b4eec64 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:00.949398 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06794b57-5606-406d-965e-e6602b4eec64 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:00.949696 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06794b57-5606-406d-965e-e6602b4eec64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:00.975459 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-06794b57-5606-406d-965e-e6602b4eec64 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:33:00.976965 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-06794b57-5606-406d-965e-e6602b4eec64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:33:00.996010 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:01.000543 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf83ceb6-2a6b-4690-96da-d1960690c7db None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:01.037869 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6487ba7-9bf0-42bb-876a-1db4d61856f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:01.038505 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6487ba7-9bf0-42bb-876a-1db4d61856f6 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:01.038841 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6487ba7-9bf0-42bb-876a-1db4d61856f6 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:01.039116 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6487ba7-9bf0-42bb-876a-1db4d61856f6 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:01.039694 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6487ba7-9bf0-42bb-876a-1db4d61856f6 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:01.041551 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6487ba7-9bf0-42bb-876a-1db4d61856f6 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:01.051500 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-c6487ba7-9bf0-42bb-876a-1db4d61856f6 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 15 08:33:01.051500 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:33:01.051500 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 08:33:01.051500 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 08:33:01.051500 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 08:33:01.051500 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 08:33:01.051500 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 08:33:01.051500 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 08:33:01.051500 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 15 08:33:01.051500 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:33:01.051500 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 08:33:01.051500 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:33:01.051500 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:33:01.051500 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:33:01.051500 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:33:01.051500 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:33:01.051500 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:33:01.052527 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:33:01.052527 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:33:01.052527 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:33:01.052527 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:33:01.052527 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:33:01.052527 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:33:01.052527 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 08:33:01.052527 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 08:33:01.052527 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 08:33:01.052527 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 08:33:01.052527 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:33:01.052527 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:33:01.052527 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:33:01.052527 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:33:01.052527 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:33:01.052527 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:33:01.053372 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:33:01.053372 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:33:01.053372 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:33:01.053372 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:33:01.053372 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:33:01.053372 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:33:01.053372 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:33:01.053372 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:33:01.053372 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:33:01.053372 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:33:01.053372 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 08:33:01.053372 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 08:33:01.053372 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 15 08:33:01.053372 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:33:01.053372 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 361/725] 10.222.0.30 () {56 vars in 1114 bytes} [Sat Oct 15 08:33:00 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:33:01.054201 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 364/725] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:33:00 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 08:33:01.062630 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6487ba7-9bf0-42bb-876a-1db4d61856f6 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:01.173792 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6487ba7-9bf0-42bb-876a-1db4d61856f6 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:01.209362 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48c7431-07a4-4bf0-9e03-648073b09cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:01.209929 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c7431-07a4-4bf0-9e03-648073b09cce None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:01.210213 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c7431-07a4-4bf0-9e03-648073b09cce None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:01.210491 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c7431-07a4-4bf0-9e03-648073b09cce None admin] PATH_INFO: `/v3/services` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:01.211154 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48c7431-07a4-4bf0-9e03-648073b09cce None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:33:01.211461 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48c7431-07a4-4bf0-9e03-648073b09cce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:01.212848 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48c7431-07a4-4bf0-9e03-648073b09cce None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:01.218249 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 362/726] 10.222.0.30 () {56 vars in 1114 bytes} [Sat Oct 15 08:33:01 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:33:01.223783 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06794b57-5606-406d-965e-e6602b4eec64 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:01.320879 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:01.330775 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06794b57-5606-406d-965e-e6602b4eec64 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:01.332185 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48c7431-07a4-4bf0-9e03-648073b09cce None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:01.366291 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:01.368801 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38130c03-1e77-4c4f-9e89-81ea8fc0a6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:01.369431 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38130c03-1e77-4c4f-9e89-81ea8fc0a6c1 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:01.369734 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38130c03-1e77-4c4f-9e89-81ea8fc0a6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:01.370015 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38130c03-1e77-4c4f-9e89-81ea8fc0a6c1 None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:01.370661 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38130c03-1e77-4c4f-9e89-81ea8fc0a6c1 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:33:01.371072 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38130c03-1e77-4c4f-9e89-81ea8fc0a6c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:01.372357 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38130c03-1e77-4c4f-9e89-81ea8fc0a6c1 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:01.381717 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 365/727] 10.222.0.30 () {56 vars in 1114 bytes} [Sat Oct 15 08:33:01 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 08:33:01.384678 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38130c03-1e77-4c4f-9e89-81ea8fc0a6c1 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:01.454041 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48c7431-07a4-4bf0-9e03-648073b09cce None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:01.487525 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159d29db-b311-45a4-bcf0-35d0b5734e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:01.488473 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d29db-b311-45a4-bcf0-35d0b5734e08 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:01.488975 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d29db-b311-45a4-bcf0-35d0b5734e08 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:01.489442 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d29db-b311-45a4-bcf0-35d0b5734e08 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:01.490269 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159d29db-b311-45a4-bcf0-35d0b5734e08 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:01.492431 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159d29db-b311-45a4-bcf0-35d0b5734e08 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:01.501346 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-159d29db-b311-45a4-bcf0-35d0b5734e08 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 15 08:33:01.501346 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:33:01.501346 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 08:33:01.501346 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 08:33:01.501346 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 08:33:01.501346 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 08:33:01.501346 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 08:33:01.501346 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 08:33:01.501346 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 15 08:33:01.501346 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:33:01.501346 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 08:33:01.501346 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:33:01.501346 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:33:01.501346 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:33:01.501346 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:33:01.501346 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:33:01.501346 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:33:01.502553 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:33:01.502553 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:33:01.502553 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:33:01.502553 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:33:01.502553 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:33:01.502553 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:33:01.502553 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 08:33:01.502553 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 08:33:01.502553 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 08:33:01.502553 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 08:33:01.502553 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:33:01.502553 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:33:01.502553 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:33:01.502553 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:33:01.502553 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:33:01.502553 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:33:01.503385 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:33:01.503385 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:33:01.503385 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:33:01.503385 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:33:01.503385 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:33:01.503385 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:33:01.503385 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:33:01.503385 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:33:01.503385 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:33:01.503385 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:33:01.503385 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 08:33:01.503385 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 08:33:01.503385 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 15 08:33:01.503385 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:33:01.504138 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38130c03-1e77-4c4f-9e89-81ea8fc0a6c1 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:01.506111 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 363/728] 10.222.0.30 () {56 vars in 1126 bytes} [Sat Oct 15 08:33:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 08:33:01.514014 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159d29db-b311-45a4-bcf0-35d0b5734e08 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:01.546637 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac44b94-8be6-44b5-8963-5141265e7a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:01.547521 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac44b94-8be6-44b5-8963-5141265e7a7f None admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:01.547805 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac44b94-8be6-44b5-8963-5141265e7a7f None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:01.548533 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac44b94-8be6-44b5-8963-5141265e7a7f None admin] PATH_INFO: `/v3/endpoints` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:01.549491 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac44b94-8be6-44b5-8963-5141265e7a7f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:01.551073 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac44b94-8be6-44b5-8963-5141265e7a7f None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:01.581967 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 366/729] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:33:01 2022] POST /identity/v3/endpoints => generated 346 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:33:01.633842 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:01.644198 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159d29db-b311-45a4-bcf0-35d0b5734e08 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:01.690858 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de131cf-70e4-46cf-a896-dd84425f1965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:01.691490 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de131cf-70e4-46cf-a896-dd84425f1965 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:01.691774 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de131cf-70e4-46cf-a896-dd84425f1965 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:01.692047 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de131cf-70e4-46cf-a896-dd84425f1965 None admin] PATH_INFO: `/v3/services` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:01.692686 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de131cf-70e4-46cf-a896-dd84425f1965 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:33:01.692994 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de131cf-70e4-46cf-a896-dd84425f1965 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:01.694424 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de131cf-70e4-46cf-a896-dd84425f1965 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:01.700572 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 364/730] 10.222.0.30 () {56 vars in 1122 bytes} [Sat Oct 15 08:33:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:33:01.706874 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac44b94-8be6-44b5-8963-5141265e7a7f None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:01.721799 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:01.822943 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac44b94-8be6-44b5-8963-5141265e7a7f None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:01.859585 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574e8999-864e-4de7-beb9-a0b80e3343f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:01.860192 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e8999-864e-4de7-beb9-a0b80e3343f1 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:01.860474 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e8999-864e-4de7-beb9-a0b80e3343f1 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:01.860749 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e8999-864e-4de7-beb9-a0b80e3343f1 None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:01.861377 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574e8999-864e-4de7-beb9-a0b80e3343f1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:33:01.861698 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574e8999-864e-4de7-beb9-a0b80e3343f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:01.863151 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574e8999-864e-4de7-beb9-a0b80e3343f1 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:01.868560 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 367/731] 10.222.0.30 () {56 vars in 1122 bytes} [Sat Oct 15 08:33:01 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:33:01.874752 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de131cf-70e4-46cf-a896-dd84425f1965 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:01.983831 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de131cf-70e4-46cf-a896-dd84425f1965 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:02.020073 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d5a7ec-a5a8-4f21-b05b-d22fd4d6024f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:02.020688 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d5a7ec-a5a8-4f21-b05b-d22fd4d6024f None admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:02.021019 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d5a7ec-a5a8-4f21-b05b-d22fd4d6024f None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:02.021303 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d5a7ec-a5a8-4f21-b05b-d22fd4d6024f None admin] PATH_INFO: `/v3/endpoints` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:02.022067 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d5a7ec-a5a8-4f21-b05b-d22fd4d6024f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:02.023514 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d5a7ec-a5a8-4f21-b05b-d22fd4d6024f None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:02.048248 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 365/732] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:33:01 2022] POST /identity/v3/endpoints => generated 349 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:33:02.699861 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be603968-3b55-4e50-b501-b80ce4acf38e None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:02.700870 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be603968-3b55-4e50-b501-b80ce4acf38e None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:02.701363 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be603968-3b55-4e50-b501-b80ce4acf38e None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:02.703172 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 368/733] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:33:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:33:02.708676 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3dda05-084b-4f6a-a562-50d1d83a978a None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:02.709184 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3dda05-084b-4f6a-a562-50d1d83a978a None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:02.709708 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3dda05-084b-4f6a-a562-50d1d83a978a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:02.723708 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:02.730462 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-de3dda05-084b-4f6a-a562-50d1d83a978a None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:33:02.731793 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-de3dda05-084b-4f6a-a562-50d1d83a978a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:33:02.811462 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 366/734] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:33:02 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 08:33:02.918134 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b80c42d-d39f-4a16-aaae-3d51fb669cd5 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:02.918421 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b80c42d-d39f-4a16-aaae-3d51fb669cd5 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:02.918703 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b80c42d-d39f-4a16-aaae-3d51fb669cd5 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:02.920289 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 369/735] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:33:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:33:02.925660 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9edad-eb6f-4930-b8c5-c8063e155b7c None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:02.926221 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9edad-eb6f-4930-b8c5-c8063e155b7c None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:02.926708 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9edad-eb6f-4930-b8c5-c8063e155b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:02.946027 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-5ac9edad-eb6f-4930-b8c5-c8063e155b7c None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:33:02.947664 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-5ac9edad-eb6f-4930-b8c5-c8063e155b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:33:03.018642 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 367/736] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:33:02 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 08:33:03.092052 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b80c42d-d39f-4a16-aaae-3d51fb669cd5 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:03.173207 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:03.200525 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b80c42d-d39f-4a16-aaae-3d51fb669cd5 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:03.234183 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98516077-7fae-4958-b333-4bd4d7ed4672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:03.235153 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98516077-7fae-4958-b333-4bd4d7ed4672 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:03.235487 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98516077-7fae-4958-b333-4bd4d7ed4672 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:03.235767 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98516077-7fae-4958-b333-4bd4d7ed4672 None admin] PATH_INFO: `/v3/services/compute` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:03.236343 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98516077-7fae-4958-b333-4bd4d7ed4672 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:03.237712 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98516077-7fae-4958-b333-4bd4d7ed4672 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:03.247317 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-98516077-7fae-4958-b333-4bd4d7ed4672 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 15 08:33:03.247317 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:33:03.247317 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 08:33:03.247317 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 08:33:03.247317 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 08:33:03.247317 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 08:33:03.247317 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 08:33:03.247317 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 08:33:03.247317 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 15 08:33:03.247317 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:33:03.247317 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 08:33:03.247317 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:33:03.247317 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:33:03.247317 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:33:03.247317 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:33:03.247317 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:33:03.247317 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:33:03.248320 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:33:03.248320 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:33:03.248320 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:33:03.248320 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:33:03.248320 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:33:03.248320 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:33:03.248320 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 08:33:03.248320 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 08:33:03.248320 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 08:33:03.248320 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 08:33:03.248320 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:33:03.248320 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:33:03.248320 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:33:03.248320 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:33:03.248320 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:33:03.248320 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:33:03.249189 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:33:03.249189 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:33:03.249189 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:33:03.249189 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:33:03.249189 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:33:03.249189 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:33:03.249189 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:33:03.249189 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:33:03.249189 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:33:03.249189 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:33:03.249189 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 08:33:03.249189 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 08:33:03.249189 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 15 08:33:03.249189 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:33:03.249189 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 370/737] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 08:33:03 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:33:03.255001 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac9edad-eb6f-4930-b8c5-c8063e155b7c None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:03.309167 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98516077-7fae-4958-b333-4bd4d7ed4672 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:03.317171 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:03.362207 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac9edad-eb6f-4930-b8c5-c8063e155b7c None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:03.411830 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3340fa-561b-4862-9c31-1c2e07ec5c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:03.413077 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3340fa-561b-4862-9c31-1c2e07ec5c03 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:03.414430 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3340fa-561b-4862-9c31-1c2e07ec5c03 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:03.414724 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3340fa-561b-4862-9c31-1c2e07ec5c03 None admin] PATH_INFO: `/v3/services` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:03.415368 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3340fa-561b-4862-9c31-1c2e07ec5c03 None admin] RBAC: Adding query filter params (name=compute) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:33:03.415673 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3340fa-561b-4862-9c31-1c2e07ec5c03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:03.417061 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3340fa-561b-4862-9c31-1c2e07ec5c03 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:03.426792 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98516077-7fae-4958-b333-4bd4d7ed4672 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:03.427108 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 368/738] 10.222.0.30 () {56 vars in 1112 bytes} [Sat Oct 15 08:33:03 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:33:03.427979 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3340fa-561b-4862-9c31-1c2e07ec5c03 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:03.467361 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf2fa3a-4607-4600-84a9-9b39e2aa9df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:03.467973 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2fa3a-4607-4600-84a9-9b39e2aa9df2 None admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:03.468255 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2fa3a-4607-4600-84a9-9b39e2aa9df2 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:03.468533 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2fa3a-4607-4600-84a9-9b39e2aa9df2 None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:03.469220 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf2fa3a-4607-4600-84a9-9b39e2aa9df2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:03.470621 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf2fa3a-4607-4600-84a9-9b39e2aa9df2 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:03.481280 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:03.492820 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 371/739] 10.222.0.30 () {60 vars in 1141 bytes} [Sat Oct 15 08:33:03 2022] POST /identity/v3/services => generated 242 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 08:33:03.553357 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3340fa-561b-4862-9c31-1c2e07ec5c03 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:03.584632 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:03.601884 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cfbbc2-a883-4046-92cd-39b184bc12bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:03.602885 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cfbbc2-a883-4046-92cd-39b184bc12bc None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:03.603370 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cfbbc2-a883-4046-92cd-39b184bc12bc None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:03.603845 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cfbbc2-a883-4046-92cd-39b184bc12bc None admin] PATH_INFO: `/v3/services` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:03.604743 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cfbbc2-a883-4046-92cd-39b184bc12bc None admin] RBAC: Adding query filter params (type=compute) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:33:03.605292 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cfbbc2-a883-4046-92cd-39b184bc12bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:03.606937 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cfbbc2-a883-4046-92cd-39b184bc12bc None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:03.613434 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 369/740] 10.222.0.30 () {56 vars in 1112 bytes} [Sat Oct 15 08:33:03 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:33:03.624581 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf2fa3a-4607-4600-84a9-9b39e2aa9df2 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:03.729319 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf2fa3a-4607-4600-84a9-9b39e2aa9df2 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:03.766117 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86334f68-f2e7-43ed-bd0b-9e424960bcbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:03.767409 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86334f68-f2e7-43ed-bd0b-9e424960bcbb None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:03.767899 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86334f68-f2e7-43ed-bd0b-9e424960bcbb None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:03.768397 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86334f68-f2e7-43ed-bd0b-9e424960bcbb None admin] PATH_INFO: `/v3/endpoints` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:03.769436 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86334f68-f2e7-43ed-bd0b-9e424960bcbb None admin] RBAC: Adding query filter params (interface=public, service_id=1497a3cd8b854c72af5f9aa23fa752c9, region_id=RegionOne) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:33:03.769993 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86334f68-f2e7-43ed-bd0b-9e424960bcbb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:03.771623 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86334f68-f2e7-43ed-bd0b-9e424960bcbb None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:03.779136 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 372/741] 10.222.0.30 () {56 vars in 1251 bytes} [Sat Oct 15 08:33:03 2022] GET /identity/v3/endpoints?service_id=1497a3cd8b854c72af5f9aa23fa752c9&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:33:03.786210 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cfbbc2-a883-4046-92cd-39b184bc12bc None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:03.894848 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9adfba4-3197-478f-88ca-7317fb69d7fa None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:03.895495 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9adfba4-3197-478f-88ca-7317fb69d7fa None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:03.895909 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cfbbc2-a883-4046-92cd-39b184bc12bc None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:03.896194 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9adfba4-3197-478f-88ca-7317fb69d7fa None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:03.898791 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 373/742] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:33:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:33:03.906349 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d8eba6-d48f-4e8a-be41-87505aa9bc28 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:03.906658 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d8eba6-d48f-4e8a-be41-87505aa9bc28 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:03.906905 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d8eba6-d48f-4e8a-be41-87505aa9bc28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:03.927361 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-55d8eba6-d48f-4e8a-be41-87505aa9bc28 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:33:03.928724 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-55d8eba6-d48f-4e8a-be41-87505aa9bc28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:33:03.940221 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9bb160-830b-4c8c-81d9-8b1937c1acd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:03.941603 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9bb160-830b-4c8c-81d9-8b1937c1acd6 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:03.942121 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9bb160-830b-4c8c-81d9-8b1937c1acd6 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:03.943016 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9bb160-830b-4c8c-81d9-8b1937c1acd6 None admin] PATH_INFO: `/v3/services` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:03.944112 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9bb160-830b-4c8c-81d9-8b1937c1acd6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:03.946486 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9bb160-830b-4c8c-81d9-8b1937c1acd6 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:03.959159 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 370/743] 10.222.0.30 () {56 vars in 1087 bytes} [Sat Oct 15 08:33:03 2022] GET /identity/v3/services => generated 1957 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:33:03.999409 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:04.011806 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 374/744] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:33:03 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 08:33:04.271948 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3da30bd-0c3e-4aff-a13e-53035cb7974a None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:04.272463 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3da30bd-0c3e-4aff-a13e-53035cb7974a None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:04.272571 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3da30bd-0c3e-4aff-a13e-53035cb7974a None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:04.274061 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 371/745] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:33:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:33:04.282705 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af571b-e40f-43c5-af2f-e4f25a38db33 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:04.282705 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af571b-e40f-43c5-af2f-e4f25a38db33 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:04.282705 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af571b-e40f-43c5-af2f-e4f25a38db33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:04.298425 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3da30bd-0c3e-4aff-a13e-53035cb7974a None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:04.299516 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-92af571b-e40f-43c5-af2f-e4f25a38db33 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:33:04.300904 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-92af571b-e40f-43c5-af2f-e4f25a38db33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:33:04.365144 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:04.373162 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 375/746] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:33:04 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 08:33:04.427745 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3da30bd-0c3e-4aff-a13e-53035cb7974a None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:04.465636 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb279e6-4b2d-49b1-8418-d55d529137b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:04.466315 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb279e6-4b2d-49b1-8418-d55d529137b4 None admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:04.466605 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb279e6-4b2d-49b1-8418-d55d529137b4 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:04.466915 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb279e6-4b2d-49b1-8418-d55d529137b4 None admin] PATH_INFO: `/v3/projects` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:04.467587 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb279e6-4b2d-49b1-8418-d55d529137b4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:04.469016 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb279e6-4b2d-49b1-8418-d55d529137b4 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:04.471732 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: WARNING keystone.server.flask.common [None req-8eb279e6-4b2d-49b1-8418-d55d529137b4 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 15 08:33:04.491200 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.sql.core [None req-8eb279e6-4b2d-49b1-8418-d55d529137b4 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 15 08:33:04.491200 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 15 08:33:04.491200 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [parameters: {'id': 'f0a3863596114e17a992eba6caebdb21', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 15 08:33:04.491200 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88060) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 15 08:33:04.510149 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-8eb279e6-4b2d-49b1-8418-d55d529137b4 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f0a3863596114e17a992eba6caebdb21.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f0a3863596114e17a992eba6caebdb21. Oct 15 08:33:04.510149 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:33:04.510149 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 15 08:33:04.510149 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 15 08:33:04.510149 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 15 08:33:04.510149 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 15 08:33:04.510149 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 15 08:33:04.510149 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application result = self._query(query) Oct 15 08:33:04.510149 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 15 08:33:04.510149 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application conn.query(q) Oct 15 08:33:04.510149 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 15 08:33:04.510149 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 15 08:33:04.510149 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 15 08:33:04.510989 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application result.read() Oct 15 08:33:04.510989 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 15 08:33:04.510989 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 15 08:33:04.510989 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 15 08:33:04.510989 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 15 08:33:04.510989 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 15 08:33:04.510989 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 15 08:33:04.510989 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 15 08:33:04.510989 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 15 08:33:04.510989 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 15 08:33:04.510989 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:33:04.510989 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 15 08:33:04.510989 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:33:04.510989 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:33:04.510989 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 15 08:33:04.510989 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 15 08:33:04.510989 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 15 08:33:04.510989 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 15 08:33:04.511928 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 15 08:33:04.511928 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application next(self.gen) Oct 15 08:33:04.511928 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 15 08:33:04.511928 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application yield resource Oct 15 08:33:04.511928 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 15 08:33:04.511928 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application next(self.gen) Oct 15 08:33:04.511928 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 15 08:33:04.511928 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application self.session.rollback() Oct 15 08:33:04.511928 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 08:33:04.511928 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application self.force_reraise() Oct 15 08:33:04.511928 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 08:33:04.511928 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise self.value Oct 15 08:33:04.511928 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 15 08:33:04.511928 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 15 08:33:04.511928 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 15 08:33:04.511928 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application session.commit() Oct 15 08:33:04.511928 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 15 08:33:04.512838 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 15 08:33:04.512838 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 15 08:33:04.512838 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application self._prepare_impl() Oct 15 08:33:04.512838 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 15 08:33:04.512838 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application self.session.flush() Oct 15 08:33:04.512838 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 15 08:33:04.512838 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application self._flush(objects) Oct 15 08:33:04.512838 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 15 08:33:04.512838 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 15 08:33:04.512838 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 15 08:33:04.512838 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application compat.raise_( Oct 15 08:33:04.512838 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 15 08:33:04.512838 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception Oct 15 08:33:04.512838 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 15 08:33:04.512838 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application flush_context.execute() Oct 15 08:33:04.512838 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 15 08:33:04.512838 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rec.execute(self) Oct 15 08:33:04.513737 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 15 08:33:04.513737 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 15 08:33:04.513737 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 15 08:33:04.513737 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 15 08:33:04.513737 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 15 08:33:04.513737 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 15 08:33:04.513737 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 15 08:33:04.513737 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 15 08:33:04.513737 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 15 08:33:04.513737 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 15 08:33:04.513737 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 15 08:33:04.513737 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 15 08:33:04.513737 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 15 08:33:04.513737 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 15 08:33:04.513737 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 15 08:33:04.514561 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 15 08:33:04.514561 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 15 08:33:04.514561 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception Oct 15 08:33:04.514561 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 15 08:33:04.514561 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 15 08:33:04.514561 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 15 08:33:04.514561 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 15 08:33:04.514561 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 15 08:33:04.514561 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application result = self._query(query) Oct 15 08:33:04.514561 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 15 08:33:04.514561 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application conn.query(q) Oct 15 08:33:04.514561 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 15 08:33:04.514561 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 15 08:33:04.514561 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 15 08:33:04.514561 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application result.read() Oct 15 08:33:04.514561 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 15 08:33:04.514561 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 15 08:33:04.515427 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 15 08:33:04.515427 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 15 08:33:04.515427 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 15 08:33:04.515427 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 15 08:33:04.515427 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 15 08:33:04.515427 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 15 08:33:04.515427 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 15 08:33:04.515427 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 15 08:33:04.515427 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [parameters: {'id': 'f0a3863596114e17a992eba6caebdb21', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 15 08:33:04.515427 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 15 08:33:04.515427 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:33:04.515427 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 08:33:04.515427 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:33:04.515427 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:33:04.515427 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 15 08:33:04.516203 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 15 08:33:04.516203 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 15 08:33:04.516203 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 15 08:33:04.516203 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 15 08:33:04.516203 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:33:04.516203 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 08:33:04.516203 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:33:04.516203 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:33:04.516203 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:33:04.516203 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:33:04.516203 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:33:04.516203 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:33:04.516203 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:33:04.516203 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:33:04.516203 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:33:04.516203 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:33:04.516203 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:33:04.519408 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:33:04.519408 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 15 08:33:04.519408 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 15 08:33:04.519408 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:33:04.519408 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:33:04.519408 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 15 08:33:04.519408 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 15 08:33:04.519408 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f0a3863596114e17a992eba6caebdb21. Oct 15 08:33:04.519408 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:33:04.519408 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 372/747] 10.222.0.30 () {60 vars in 1140 bytes} [Sat Oct 15 08:33:04 2022] POST /identity/v3/projects => generated 262 bytes in 214 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 15 08:33:04.520664 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92af571b-e40f-43c5-af2f-e4f25a38db33 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:04.623529 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92af571b-e40f-43c5-af2f-e4f25a38db33 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:04.648687 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:04.659843 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9db908-e4c9-4bc1-9103-799f803765d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:04.660538 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9db908-e4c9-4bc1-9103-799f803765d0 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:04.660832 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9db908-e4c9-4bc1-9103-799f803765d0 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:04.661113 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9db908-e4c9-4bc1-9103-799f803765d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:04.665521 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb279e6-4b2d-49b1-8418-d55d529137b4 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:04.668232 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9db908-e4c9-4bc1-9103-799f803765d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:04.670962 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9db908-e4c9-4bc1-9103-799f803765d0 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:04.686120 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-ac9db908-e4c9-4bc1-9103-799f803765d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 08:33:04.686120 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:33:04.686120 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:33:04.686120 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:33:04.686120 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:33:04.686120 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:33:04.686120 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:33:04.686120 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:33:04.686120 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:33:04.686120 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:33:04.686120 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:33:04.686120 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:33:04.686120 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 08:33:04.686120 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 08:33:04.686120 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 08:33:04.687253 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 08:33:04.687253 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:33:04.687253 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:33:04.687253 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:33:04.687253 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:33:04.687253 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:33:04.687253 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:33:04.687253 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:33:04.687253 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:33:04.687253 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:33:04.687253 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:33:04.687253 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:33:04.687253 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:33:04.687253 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:33:04.687253 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:33:04.687253 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:33:04.687253 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:33:04.688451 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 08:33:04.688451 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 08:33:04.688451 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 08:33:04.688451 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 08:33:04.688451 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 08:33:04.688451 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 08:33:04.688451 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 08:33:04.688451 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:33:04.688451 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 376/748] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 08:33:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:33:04.693221 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9db908-e4c9-4bc1-9103-799f803765d0 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:04.775588 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb279e6-4b2d-49b1-8418-d55d529137b4 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:04.782956 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:04.810445 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9db908-e4c9-4bc1-9103-799f803765d0 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:04.815242 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dbc939-4698-4942-b57a-4890bcbc5ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:04.816145 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dbc939-4698-4942-b57a-4890bcbc5ede None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:04.816621 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dbc939-4698-4942-b57a-4890bcbc5ede None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:04.817083 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dbc939-4698-4942-b57a-4890bcbc5ede None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:04.817887 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dbc939-4698-4942-b57a-4890bcbc5ede None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:04.819752 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dbc939-4698-4942-b57a-4890bcbc5ede None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:04.829119 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-41dbc939-4698-4942-b57a-4890bcbc5ede None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 15 08:33:04.829119 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:33:04.829119 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 08:33:04.829119 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 08:33:04.829119 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 08:33:04.829119 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 08:33:04.829119 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 08:33:04.829119 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 08:33:04.829119 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 15 08:33:04.829119 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:33:04.829119 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 08:33:04.829119 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:33:04.829119 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:33:04.829119 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:33:04.829119 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:33:04.829119 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:33:04.829119 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:33:04.830117 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:33:04.830117 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:33:04.830117 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:33:04.830117 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:33:04.830117 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:33:04.830117 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:33:04.830117 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 08:33:04.830117 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 08:33:04.830117 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 08:33:04.830117 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 08:33:04.830117 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:33:04.830117 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:33:04.830117 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:33:04.830117 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:33:04.830117 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:33:04.830117 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:33:04.830958 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:33:04.830958 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:33:04.830958 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:33:04.830958 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:33:04.830958 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:33:04.830958 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:33:04.830958 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:33:04.830958 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:33:04.830958 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:33:04.830958 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:33:04.830958 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 08:33:04.830958 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 08:33:04.830958 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 15 08:33:04.830958 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:33:04.838543 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 373/749] 10.222.0.30 () {56 vars in 1126 bytes} [Sat Oct 15 08:33:04 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 08:33:04.839646 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dbc939-4698-4942-b57a-4890bcbc5ede None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:04.846193 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b139d50-d3a7-476b-8bbd-79fdd23497ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:04.846803 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b139d50-d3a7-476b-8bbd-79fdd23497ca None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:04.847090 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b139d50-d3a7-476b-8bbd-79fdd23497ca None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:04.847375 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b139d50-d3a7-476b-8bbd-79fdd23497ca None admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:04.848045 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b139d50-d3a7-476b-8bbd-79fdd23497ca None admin] RBAC: Adding query filter params (name=service) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:33:04.848352 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b139d50-d3a7-476b-8bbd-79fdd23497ca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:04.849811 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b139d50-d3a7-476b-8bbd-79fdd23497ca None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:04.873896 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 377/750] 10.222.0.30 () {56 vars in 1112 bytes} [Sat Oct 15 08:33:04 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 08:33:04.964815 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dbc939-4698-4942-b57a-4890bcbc5ede None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:05.009601 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f0a0ce-fa5e-4aff-9114-56d675faa430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:05.010714 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f0a0ce-fa5e-4aff-9114-56d675faa430 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:05.011257 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f0a0ce-fa5e-4aff-9114-56d675faa430 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:05.011757 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f0a0ce-fa5e-4aff-9114-56d675faa430 None admin] PATH_INFO: `/v3/services` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:05.012652 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f0a0ce-fa5e-4aff-9114-56d675faa430 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:33:05.013176 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f0a0ce-fa5e-4aff-9114-56d675faa430 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:05.014869 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f0a0ce-fa5e-4aff-9114-56d675faa430 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:05.024371 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 374/751] 10.222.0.30 () {56 vars in 1122 bytes} [Sat Oct 15 08:33:04 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:33:05.030714 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b139d50-d3a7-476b-8bbd-79fdd23497ca None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:05.078288 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:05.135041 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b139d50-d3a7-476b-8bbd-79fdd23497ca None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:05.171390 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be930f-a6da-41cf-8c5b-f4be465e11e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:05.172277 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be930f-a6da-41cf-8c5b-f4be465e11e4 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:05.172697 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be930f-a6da-41cf-8c5b-f4be465e11e4 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:05.173114 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be930f-a6da-41cf-8c5b-f4be465e11e4 None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:05.173989 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17be930f-a6da-41cf-8c5b-f4be465e11e4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:33:05.174448 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17be930f-a6da-41cf-8c5b-f4be465e11e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:05.176054 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17be930f-a6da-41cf-8c5b-f4be465e11e4 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:05.181577 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 378/752] 10.222.0.30 () {56 vars in 1122 bytes} [Sat Oct 15 08:33:05 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:33:05.192831 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f0a0ce-fa5e-4aff-9114-56d675faa430 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:05.230499 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:05.300143 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f0a0ce-fa5e-4aff-9114-56d675faa430 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:05.336239 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ef2a6e-b1a4-4bd2-bd05-1d576b4dc797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:05.337097 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef2a6e-b1a4-4bd2-bd05-1d576b4dc797 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:05.337520 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef2a6e-b1a4-4bd2-bd05-1d576b4dc797 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:05.337983 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef2a6e-b1a4-4bd2-bd05-1d576b4dc797 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:05.338858 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ef2a6e-b1a4-4bd2-bd05-1d576b4dc797 None admin] RBAC: Adding query filter params (interface=admin, service_id=fc0069003c8e49f892e5afb67ea77d16, region_id=RegionOne) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:33:05.339322 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ef2a6e-b1a4-4bd2-bd05-1d576b4dc797 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:05.340821 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ef2a6e-b1a4-4bd2-bd05-1d576b4dc797 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:05.346521 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 375/753] 10.222.0.30 () {56 vars in 1249 bytes} [Sat Oct 15 08:33:05 2022] GET /identity/v3/endpoints?service_id=fc0069003c8e49f892e5afb67ea77d16&interface=admin®ion_id=RegionOne => generated 193 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:33:05.352443 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be930f-a6da-41cf-8c5b-f4be465e11e4 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:05.467022 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be930f-a6da-41cf-8c5b-f4be465e11e4 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:05.478396 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:05.502598 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13436e8-5fac-4086-9a18-63a9cdbc8989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:05.503354 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13436e8-5fac-4086-9a18-63a9cdbc8989 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:05.503738 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13436e8-5fac-4086-9a18-63a9cdbc8989 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:05.504111 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13436e8-5fac-4086-9a18-63a9cdbc8989 None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:05.504817 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13436e8-5fac-4086-9a18-63a9cdbc8989 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:05.506304 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13436e8-5fac-4086-9a18-63a9cdbc8989 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:05.513425 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 379/754] 10.222.0.30 () {56 vars in 1087 bytes} [Sat Oct 15 08:33:05 2022] GET /identity/v3/services => generated 1957 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:33:05.745671 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b9d9c-8176-477c-a0ed-71e6ab18fa90 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:05.745996 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b9d9c-8176-477c-a0ed-71e6ab18fa90 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:05.746370 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b9d9c-8176-477c-a0ed-71e6ab18fa90 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:05.747986 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 376/755] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:33:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:33:05.753303 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99779661-f2eb-46fe-ad7e-7347ad27dd6e None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:05.753594 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99779661-f2eb-46fe-ad7e-7347ad27dd6e None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:05.753916 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99779661-f2eb-46fe-ad7e-7347ad27dd6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:05.773158 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-99779661-f2eb-46fe-ad7e-7347ad27dd6e None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:33:05.774393 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-99779661-f2eb-46fe-ad7e-7347ad27dd6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:33:05.843932 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 380/756] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:33:05 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 08:33:06.119350 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431b9d9c-8176-477c-a0ed-71e6ab18fa90 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:06.210099 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77662451-c38d-4dee-8a6d-441e1aebd1e6 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:06.210389 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77662451-c38d-4dee-8a6d-441e1aebd1e6 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:06.210728 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77662451-c38d-4dee-8a6d-441e1aebd1e6 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:06.212347 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 381/757] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:33:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:33:06.217714 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe97fe-339c-4d95-804c-0c24305dd626 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:06.218017 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe97fe-339c-4d95-804c-0c24305dd626 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:06.218299 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe97fe-339c-4d95-804c-0c24305dd626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:06.222477 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:06.225881 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431b9d9c-8176-477c-a0ed-71e6ab18fa90 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:06.238529 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-6dfe97fe-339c-4d95-804c-0c24305dd626 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:33:06.239726 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-6dfe97fe-339c-4d95-804c-0c24305dd626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:33:06.269289 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4142cf75-3f98-41f6-bd2c-f75c334e9e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:06.270442 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4142cf75-3f98-41f6-bd2c-f75c334e9e5b None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:06.270938 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4142cf75-3f98-41f6-bd2c-f75c334e9e5b None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:06.271416 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4142cf75-3f98-41f6-bd2c-f75c334e9e5b None admin] PATH_INFO: `/v3/services/image` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:06.272224 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4142cf75-3f98-41f6-bd2c-f75c334e9e5b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:06.274146 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4142cf75-3f98-41f6-bd2c-f75c334e9e5b None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:06.284059 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-4142cf75-3f98-41f6-bd2c-f75c334e9e5b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 15 08:33:06.284059 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:33:06.284059 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 08:33:06.284059 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 08:33:06.284059 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 08:33:06.284059 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 08:33:06.284059 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 08:33:06.284059 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 08:33:06.284059 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 15 08:33:06.284059 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:33:06.284059 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 08:33:06.284059 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:33:06.284059 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:33:06.284059 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:33:06.284059 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:33:06.284059 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:33:06.284059 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:33:06.285078 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:33:06.285078 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:33:06.285078 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:33:06.285078 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:33:06.285078 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:33:06.285078 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:33:06.285078 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 08:33:06.285078 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 08:33:06.285078 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 08:33:06.285078 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 08:33:06.285078 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:33:06.285078 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:33:06.285078 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:33:06.285078 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:33:06.285078 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:33:06.285078 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:33:06.285983 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:33:06.285983 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:33:06.285983 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:33:06.285983 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:33:06.285983 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:33:06.285983 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:33:06.285983 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:33:06.285983 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:33:06.285983 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:33:06.285983 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:33:06.285983 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 08:33:06.285983 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 08:33:06.285983 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 15 08:33:06.285983 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:33:06.289046 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 377/758] 10.222.0.30 () {56 vars in 1105 bytes} [Sat Oct 15 08:33:06 2022] GET /identity/v3/services/image => generated 86 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:33:06.294515 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4142cf75-3f98-41f6-bd2c-f75c334e9e5b None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:06.333701 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 382/759] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:33:06 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 08:33:06.411271 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4142cf75-3f98-41f6-bd2c-f75c334e9e5b None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:06.450912 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afaf0dc-e67e-4e82-afcc-1f0b86ea4633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:06.451524 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afaf0dc-e67e-4e82-afcc-1f0b86ea4633 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:06.451803 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afaf0dc-e67e-4e82-afcc-1f0b86ea4633 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:06.452076 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afaf0dc-e67e-4e82-afcc-1f0b86ea4633 None admin] PATH_INFO: `/v3/services` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:06.452704 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afaf0dc-e67e-4e82-afcc-1f0b86ea4633 None admin] RBAC: Adding query filter params (name=image) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:33:06.453007 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afaf0dc-e67e-4e82-afcc-1f0b86ea4633 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:06.454418 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afaf0dc-e67e-4e82-afcc-1f0b86ea4633 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:06.459692 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 378/760] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:33:06 2022] GET /identity/v3/services?name=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 08:33:06.465123 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe97fe-339c-4d95-804c-0c24305dd626 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:06.496999 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:06.571788 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe97fe-339c-4d95-804c-0c24305dd626 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:06.604998 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afaf0dc-e67e-4e82-afcc-1f0b86ea4633 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:06.606197 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1061d3f-fc7f-4afd-a700-329e409a0c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:06.606815 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1061d3f-fc7f-4afd-a700-329e409a0c44 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:06.607098 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1061d3f-fc7f-4afd-a700-329e409a0c44 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:06.607373 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1061d3f-fc7f-4afd-a700-329e409a0c44 None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:06.608008 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1061d3f-fc7f-4afd-a700-329e409a0c44 None admin] RBAC: Adding query filter params (type=image) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:33:06.608316 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1061d3f-fc7f-4afd-a700-329e409a0c44 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:06.610036 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1061d3f-fc7f-4afd-a700-329e409a0c44 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:06.617398 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 383/761] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:33:06 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:33:06.622742 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1061d3f-fc7f-4afd-a700-329e409a0c44 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:06.648916 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:06.726666 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:06.733386 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1061d3f-fc7f-4afd-a700-329e409a0c44 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:06.733742 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afaf0dc-e67e-4e82-afcc-1f0b86ea4633 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:06.774267 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4f858e-ad3e-45c0-8536-5314a4710f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:06.775387 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4f858e-ad3e-45c0-8536-5314a4710f6c None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:06.775971 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4f858e-ad3e-45c0-8536-5314a4710f6c None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:06.776466 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4f858e-ad3e-45c0-8536-5314a4710f6c None admin] PATH_INFO: `/v3/endpoints` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:06.777385 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4f858e-ad3e-45c0-8536-5314a4710f6c None admin] RBAC: Adding query filter params (interface=public, service_id=c8fabbc0c02f4dbd92168d95326c58d4, region_id=RegionOne) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:33:06.778204 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4f858e-ad3e-45c0-8536-5314a4710f6c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:06.780707 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4f858e-ad3e-45c0-8536-5314a4710f6c None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:06.784668 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba4b25-5b7b-4bc9-86b9-988df083b796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:06.785635 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba4b25-5b7b-4bc9-86b9-988df083b796 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:06.786027 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba4b25-5b7b-4bc9-86b9-988df083b796 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:06.786388 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba4b25-5b7b-4bc9-86b9-988df083b796 None admin] PATH_INFO: `/v3/services/compute` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:06.787048 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dba4b25-5b7b-4bc9-86b9-988df083b796 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:06.792712 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dba4b25-5b7b-4bc9-86b9-988df083b796 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:06.792848 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 384/762] 10.222.0.30 () {56 vars in 1251 bytes} [Sat Oct 15 08:33:06 2022] GET /identity/v3/endpoints?service_id=c8fabbc0c02f4dbd92168d95326c58d4&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 08:33:06.793587 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4f858e-ad3e-45c0-8536-5314a4710f6c None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:06.808596 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-1dba4b25-5b7b-4bc9-86b9-988df083b796 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 15 08:33:06.808596 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:33:06.808596 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 08:33:06.808596 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 08:33:06.808596 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 08:33:06.808596 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 08:33:06.808596 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 08:33:06.808596 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 08:33:06.808596 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 15 08:33:06.808596 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:33:06.808596 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 08:33:06.808596 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:33:06.808596 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:33:06.808596 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:33:06.808596 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:33:06.808596 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:33:06.808596 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:33:06.809713 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:33:06.809713 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:33:06.809713 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:33:06.809713 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:33:06.809713 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:33:06.809713 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:33:06.809713 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 08:33:06.809713 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 08:33:06.809713 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 08:33:06.809713 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 08:33:06.809713 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:33:06.809713 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:33:06.809713 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:33:06.809713 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:33:06.809713 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:33:06.809713 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:33:06.810604 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:33:06.810604 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:33:06.810604 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:33:06.810604 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:33:06.810604 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:33:06.810604 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:33:06.810604 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:33:06.810604 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:33:06.810604 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:33:06.810604 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:33:06.810604 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 08:33:06.810604 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 08:33:06.810604 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 15 08:33:06.810604 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:33:06.810604 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 379/763] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 08:33:06 2022] GET /identity/v3/services/compute => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 08:33:06.815552 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba4b25-5b7b-4bc9-86b9-988df083b796 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:06.921339 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:06.934722 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba4b25-5b7b-4bc9-86b9-988df083b796 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:06.981790 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8191db1c-e41a-4801-851e-3492c340e811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:06.982346 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8191db1c-e41a-4801-851e-3492c340e811 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:06.982629 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8191db1c-e41a-4801-851e-3492c340e811 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:06.982907 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8191db1c-e41a-4801-851e-3492c340e811 None admin] PATH_INFO: `/v3/services` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:06.983553 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8191db1c-e41a-4801-851e-3492c340e811 None admin] RBAC: Adding query filter params (name=compute) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:33:06.983859 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8191db1c-e41a-4801-851e-3492c340e811 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:06.985273 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8191db1c-e41a-4801-851e-3492c340e811 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:06.996418 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 380/764] 10.222.0.30 () {56 vars in 1112 bytes} [Sat Oct 15 08:33:06 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 08:33:06.998066 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8191db1c-e41a-4801-851e-3492c340e811 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:07.040610 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4f858e-ad3e-45c0-8536-5314a4710f6c None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:07.079690 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90389e3d-d234-4886-8dc7-83e6db21a570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:07.080308 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90389e3d-d234-4886-8dc7-83e6db21a570 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:07.080591 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90389e3d-d234-4886-8dc7-83e6db21a570 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:07.080862 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90389e3d-d234-4886-8dc7-83e6db21a570 None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:07.081474 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90389e3d-d234-4886-8dc7-83e6db21a570 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:07.082877 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90389e3d-d234-4886-8dc7-83e6db21a570 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:07.091434 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 385/765] 10.222.0.30 () {56 vars in 1087 bytes} [Sat Oct 15 08:33:06 2022] GET /identity/v3/services => generated 1957 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:33:07.118509 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8191db1c-e41a-4801-851e-3492c340e811 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:07.163277 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6361cdc-e6aa-48e5-9b58-c9e6ef9b8082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:07.164846 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6361cdc-e6aa-48e5-9b58-c9e6ef9b8082 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:07.165337 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6361cdc-e6aa-48e5-9b58-c9e6ef9b8082 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:07.165866 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6361cdc-e6aa-48e5-9b58-c9e6ef9b8082 None admin] PATH_INFO: `/v3/services` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:07.166732 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6361cdc-e6aa-48e5-9b58-c9e6ef9b8082 None admin] RBAC: Adding query filter params (type=compute) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:33:07.167286 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6361cdc-e6aa-48e5-9b58-c9e6ef9b8082 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:07.168919 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6361cdc-e6aa-48e5-9b58-c9e6ef9b8082 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:07.179675 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 381/766] 10.222.0.30 () {56 vars in 1112 bytes} [Sat Oct 15 08:33:06 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 08:33:07.182142 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90389e3d-d234-4886-8dc7-83e6db21a570 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:07.210785 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:07.268023 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db1cb7-7aaa-499f-ba36-9c810192a741 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:07.268387 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db1cb7-7aaa-499f-ba36-9c810192a741 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:07.268582 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db1cb7-7aaa-499f-ba36-9c810192a741 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:07.270778 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 382/767] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:33:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 08:33:07.275809 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c9660b-da14-4f71-b949-4bdf70dce26e None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:07.276221 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c9660b-da14-4f71-b949-4bdf70dce26e None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:07.276506 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c9660b-da14-4f71-b949-4bdf70dce26e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:07.300381 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90389e3d-d234-4886-8dc7-83e6db21a570 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:07.300573 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-67c9660b-da14-4f71-b949-4bdf70dce26e None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:33:07.302015 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-67c9660b-da14-4f71-b949-4bdf70dce26e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:33:07.317647 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:07.342695 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3188baf-d5d5-42e7-a631-c1ecec5ef75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:07.343317 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3188baf-d5d5-42e7-a631-c1ecec5ef75c None admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:07.343597 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3188baf-d5d5-42e7-a631-c1ecec5ef75c None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:07.343869 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3188baf-d5d5-42e7-a631-c1ecec5ef75c None admin] PATH_INFO: `/v3/endpoints` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:07.344557 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3188baf-d5d5-42e7-a631-c1ecec5ef75c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:07.345953 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3188baf-d5d5-42e7-a631-c1ecec5ef75c None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:07.377702 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 386/768] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:33:07 2022] POST /identity/v3/endpoints => generated 334 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:33:07.415078 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 383/769] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:33:07 2022] POST /identity/v3/auth/tokens => generated 1822 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 08:33:07.691227 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3188baf-d5d5-42e7-a631-c1ecec5ef75c None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:07.707971 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08d87a9-455d-435d-b7f1-6e7ff6ad01ba None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:07.708317 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08d87a9-455d-435d-b7f1-6e7ff6ad01ba None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:07.708527 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08d87a9-455d-435d-b7f1-6e7ff6ad01ba None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:07.710143 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 384/770] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:33:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 08:33:07.715656 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278a67df-1bba-4860-bd9c-353e062edf8d None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:07.715938 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278a67df-1bba-4860-bd9c-353e062edf8d None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:07.716211 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278a67df-1bba-4860-bd9c-353e062edf8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:07.734892 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-278a67df-1bba-4860-bd9c-353e062edf8d None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:33:07.736069 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-278a67df-1bba-4860-bd9c-353e062edf8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:33:07.783868 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:07.811186 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3188baf-d5d5-42e7-a631-c1ecec5ef75c None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:07.829771 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 385/771] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:33:07 2022] POST /identity/v3/auth/tokens => generated 1822 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 08:33:07.852472 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473f638e-6cf1-4879-980c-8e991750693a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:07.853408 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473f638e-6cf1-4879-980c-8e991750693a None admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:07.853932 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473f638e-6cf1-4879-980c-8e991750693a None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:07.854415 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473f638e-6cf1-4879-980c-8e991750693a None admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:07.855278 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473f638e-6cf1-4879-980c-8e991750693a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:07.856921 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473f638e-6cf1-4879-980c-8e991750693a None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:07.859683 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: WARNING keystone.server.flask.common [None req-473f638e-6cf1-4879-980c-8e991750693a None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 15 08:33:07.874378 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.sql.core [None req-473f638e-6cf1-4879-980c-8e991750693a None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 15 08:33:07.874378 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 15 08:33:07.874378 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [parameters: {'user_id': '88fabbe9f7fe495f8676759f339c1e1e', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 15 08:33:07.874378 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88059) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 15 08:33:07.884087 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-473f638e-6cf1-4879-980c-8e991750693a None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 15 08:33:07.884087 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:33:07.884087 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 15 08:33:07.884087 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 15 08:33:07.884087 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 15 08:33:07.884087 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 15 08:33:07.884087 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 15 08:33:07.884087 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application result = self._query(query) Oct 15 08:33:07.884087 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 15 08:33:07.884087 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application conn.query(q) Oct 15 08:33:07.884087 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 15 08:33:07.884087 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 15 08:33:07.884087 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 15 08:33:07.884087 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application result.read() Oct 15 08:33:07.884087 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 15 08:33:07.884995 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 15 08:33:07.884995 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 15 08:33:07.884995 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 15 08:33:07.884995 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 15 08:33:07.884995 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 15 08:33:07.884995 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 15 08:33:07.884995 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 15 08:33:07.884995 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 15 08:33:07.884995 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:33:07.884995 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 15 08:33:07.884995 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:33:07.884995 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:33:07.884995 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 15 08:33:07.884995 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 15 08:33:07.884995 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 15 08:33:07.884995 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 15 08:33:07.884995 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 15 08:33:07.884995 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application next(self.gen) Oct 15 08:33:07.885969 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 15 08:33:07.885969 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application yield resource Oct 15 08:33:07.885969 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 15 08:33:07.885969 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application next(self.gen) Oct 15 08:33:07.885969 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 15 08:33:07.885969 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self.session.rollback() Oct 15 08:33:07.885969 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 08:33:07.885969 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self.force_reraise() Oct 15 08:33:07.885969 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 08:33:07.885969 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise self.value Oct 15 08:33:07.885969 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 15 08:33:07.885969 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 15 08:33:07.885969 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 15 08:33:07.885969 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application session.commit() Oct 15 08:33:07.885969 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 15 08:33:07.885969 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 15 08:33:07.885969 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 15 08:33:07.886887 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._prepare_impl() Oct 15 08:33:07.886887 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 15 08:33:07.886887 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self.session.flush() Oct 15 08:33:07.886887 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 15 08:33:07.886887 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._flush(objects) Oct 15 08:33:07.886887 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 15 08:33:07.886887 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 15 08:33:07.886887 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 15 08:33:07.886887 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application compat.raise_( Oct 15 08:33:07.886887 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 15 08:33:07.886887 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception Oct 15 08:33:07.886887 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 15 08:33:07.886887 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application flush_context.execute() Oct 15 08:33:07.886887 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 15 08:33:07.886887 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rec.execute(self) Oct 15 08:33:07.886887 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 15 08:33:07.886887 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 15 08:33:07.887775 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 15 08:33:07.887775 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 15 08:33:07.887775 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 15 08:33:07.887775 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 15 08:33:07.887775 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 15 08:33:07.887775 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 15 08:33:07.887775 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 15 08:33:07.887775 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 15 08:33:07.887775 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 15 08:33:07.887775 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 15 08:33:07.887775 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 15 08:33:07.887775 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 15 08:33:07.887775 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 15 08:33:07.887775 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 15 08:33:07.887775 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 15 08:33:07.887775 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception Oct 15 08:33:07.888627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 15 08:33:07.888627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 15 08:33:07.888627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 15 08:33:07.888627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 15 08:33:07.888627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 15 08:33:07.888627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application result = self._query(query) Oct 15 08:33:07.888627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 15 08:33:07.888627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application conn.query(q) Oct 15 08:33:07.888627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 15 08:33:07.888627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 15 08:33:07.888627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 15 08:33:07.888627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application result.read() Oct 15 08:33:07.888627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 15 08:33:07.888627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 15 08:33:07.888627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 15 08:33:07.888627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 15 08:33:07.889458 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 15 08:33:07.889458 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 15 08:33:07.889458 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 15 08:33:07.889458 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 15 08:33:07.889458 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 15 08:33:07.889458 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 15 08:33:07.889458 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [parameters: {'user_id': '88fabbe9f7fe495f8676759f339c1e1e', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 15 08:33:07.889458 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 15 08:33:07.889458 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:33:07.889458 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 08:33:07.889458 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:33:07.889458 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:33:07.889458 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:33:07.889458 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:33:07.889458 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:33:07.892448 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:33:07.892448 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:33:07.892448 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:33:07.892448 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:33:07.892448 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:33:07.892448 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:33:07.892448 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:33:07.892448 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 15 08:33:07.892448 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 15 08:33:07.892448 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:33:07.892448 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:33:07.892448 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 08:33:07.892448 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 08:33:07.892448 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 08:33:07.892448 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 08:33:07.892448 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 15 08:33:07.892448 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 15 08:33:07.894570 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 15 08:33:07.894570 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 15 08:33:07.894570 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 15 08:33:07.894570 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 15 08:33:07.894570 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 15 08:33:07.894570 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:33:07.901805 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 387/772] 10.222.0.30 () {60 vars in 1131 bytes} [Sat Oct 15 08:33:07 2022] POST /identity/v3/users => generated 160 bytes in 206 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:33:07.903608 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278a67df-1bba-4860-bd9c-353e062edf8d None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:07.990414 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:08.017698 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278a67df-1bba-4860-bd9c-353e062edf8d None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:08.060388 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e4998a-cdbd-4c6d-9f27-6bb1de2fb8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:08.061396 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e4998a-cdbd-4c6d-9f27-6bb1de2fb8b9 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:08.061913 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e4998a-cdbd-4c6d-9f27-6bb1de2fb8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:08.062393 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e4998a-cdbd-4c6d-9f27-6bb1de2fb8b9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:08.071032 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e4998a-cdbd-4c6d-9f27-6bb1de2fb8b9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:08.072710 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e4998a-cdbd-4c6d-9f27-6bb1de2fb8b9 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:08.084705 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-e0e4998a-cdbd-4c6d-9f27-6bb1de2fb8b9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 15 08:33:08.084705 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:33:08.084705 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:33:08.084705 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:33:08.084705 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:33:08.084705 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:33:08.084705 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:33:08.084705 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:33:08.084705 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:33:08.084705 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:33:08.084705 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:33:08.084705 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:33:08.084705 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 08:33:08.084705 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 08:33:08.084705 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 08:33:08.084705 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 08:33:08.085789 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:33:08.085789 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:33:08.085789 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 08:33:08.085789 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 08:33:08.085789 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 08:33:08.085789 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 08:33:08.085789 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:33:08.085789 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:33:08.085789 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:33:08.085789 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:33:08.085789 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:33:08.085789 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:33:08.085789 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:33:08.085789 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:33:08.085789 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:33:08.085789 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:33:08.085789 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:33:08.085789 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:33:08.086729 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:33:08.086729 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:33:08.086729 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 08:33:08.086729 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 08:33:08.086729 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 08:33:08.086729 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 08:33:08.086729 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 08:33:08.086729 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 08:33:08.086729 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 15 08:33:08.086729 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:33:08.093763 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 386/773] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 08:33:07 2022] GET /identity/v3/users/cinder => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:33:08.095632 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473f638e-6cf1-4879-980c-8e991750693a None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:08.121759 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e4998a-cdbd-4c6d-9f27-6bb1de2fb8b9 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:08.217795 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473f638e-6cf1-4879-980c-8e991750693a None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:08.232548 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e4998a-cdbd-4c6d-9f27-6bb1de2fb8b9 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:08.256204 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf28fe7d-0a5e-4d59-abe8-0990a9f6f672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:08.256824 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf28fe7d-0a5e-4d59-abe8-0990a9f6f672 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:08.257101 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf28fe7d-0a5e-4d59-abe8-0990a9f6f672 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:08.257486 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf28fe7d-0a5e-4d59-abe8-0990a9f6f672 None admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:08.258218 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf28fe7d-0a5e-4d59-abe8-0990a9f6f672 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:33:08.258520 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf28fe7d-0a5e-4d59-abe8-0990a9f6f672 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:08.260136 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf28fe7d-0a5e-4d59-abe8-0990a9f6f672 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:08.278318 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dca5e6-7ed0-4436-a4e7-1ae36cdb5653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:08.279553 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dca5e6-7ed0-4436-a4e7-1ae36cdb5653 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:08.280068 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dca5e6-7ed0-4436-a4e7-1ae36cdb5653 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:08.280566 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dca5e6-7ed0-4436-a4e7-1ae36cdb5653 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:08.281363 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dca5e6-7ed0-4436-a4e7-1ae36cdb5653 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:08.283137 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dca5e6-7ed0-4436-a4e7-1ae36cdb5653 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:08.285167 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:08.287671 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:08.298537 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-f5dca5e6-7ed0-4436-a4e7-1ae36cdb5653 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 15 08:33:08.298537 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:33:08.298537 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 08:33:08.298537 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 08:33:08.298537 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 08:33:08.298537 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 08:33:08.298537 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 08:33:08.298537 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 08:33:08.298537 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 15 08:33:08.298537 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:33:08.298537 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 08:33:08.298537 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:33:08.298537 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:33:08.298537 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:33:08.298537 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:33:08.298537 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:33:08.298537 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:33:08.299572 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:33:08.299572 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:33:08.299572 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:33:08.299572 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:33:08.299572 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:33:08.299572 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:33:08.299572 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 08:33:08.299572 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 08:33:08.299572 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 08:33:08.299572 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 08:33:08.299572 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:33:08.299572 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:33:08.299572 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:33:08.299572 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:33:08.299572 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:33:08.299572 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:33:08.300506 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:33:08.300506 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:33:08.300506 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:33:08.300506 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:33:08.300506 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:33:08.300506 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:33:08.300506 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:33:08.300506 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:33:08.300506 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:33:08.300506 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:33:08.300506 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 08:33:08.300506 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 08:33:08.300506 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 15 08:33:08.300506 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:33:08.303293 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 387/774] 10.222.0.30 () {56 vars in 1126 bytes} [Sat Oct 15 08:33:08 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 08:33:08.309668 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dca5e6-7ed0-4436-a4e7-1ae36cdb5653 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:08.310074 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 388/775] 10.222.0.30 () {56 vars in 1101 bytes} [Sat Oct 15 08:33:08 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:33:08.428536 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dca5e6-7ed0-4436-a4e7-1ae36cdb5653 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:08.468173 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00a68a0-6388-4edb-922f-ce510620a7b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:08.468984 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00a68a0-6388-4edb-922f-ce510620a7b2 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:08.469366 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00a68a0-6388-4edb-922f-ce510620a7b2 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:08.469775 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00a68a0-6388-4edb-922f-ce510620a7b2 None admin] PATH_INFO: `/v3/services` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:08.470516 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00a68a0-6388-4edb-922f-ce510620a7b2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:33:08.470921 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00a68a0-6388-4edb-922f-ce510620a7b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:08.472415 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00a68a0-6388-4edb-922f-ce510620a7b2 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:08.478672 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 388/776] 10.222.0.30 () {56 vars in 1122 bytes} [Sat Oct 15 08:33:08 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 08:33:08.484572 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf28fe7d-0a5e-4d59-abe8-0990a9f6f672 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:08.589643 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf28fe7d-0a5e-4d59-abe8-0990a9f6f672 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:08.624578 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c567c967-05d3-42c8-aaee-52631daeedd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:08.625610 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c567c967-05d3-42c8-aaee-52631daeedd3 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:08.626122 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c567c967-05d3-42c8-aaee-52631daeedd3 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:08.626621 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c567c967-05d3-42c8-aaee-52631daeedd3 None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:08.627458 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c567c967-05d3-42c8-aaee-52631daeedd3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:33:08.627983 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c567c967-05d3-42c8-aaee-52631daeedd3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:08.629426 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c567c967-05d3-42c8-aaee-52631daeedd3 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:08.635496 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 389/777] 10.222.0.30 () {56 vars in 1122 bytes} [Sat Oct 15 08:33:08 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:33:08.641287 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00a68a0-6388-4edb-922f-ce510620a7b2 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:08.703848 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:08.744666 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00a68a0-6388-4edb-922f-ce510620a7b2 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:08.780995 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c21a1a5-464c-44ef-ab89-c2105860d5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:08.781714 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c21a1a5-464c-44ef-ab89-c2105860d5da None admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:08.782075 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c21a1a5-464c-44ef-ab89-c2105860d5da None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:08.782423 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c21a1a5-464c-44ef-ab89-c2105860d5da None admin] PATH_INFO: `/v3/endpoints` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:08.783172 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c21a1a5-464c-44ef-ab89-c2105860d5da None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:08.784617 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c21a1a5-464c-44ef-ab89-c2105860d5da None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:08.807935 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 389/778] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:33:08 2022] POST /identity/v3/endpoints => generated 325 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:33:09.369213 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccff831-4f89-4e89-a3b5-846a8373a569 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:09.369798 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccff831-4f89-4e89-a3b5-846a8373a569 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:09.369798 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccff831-4f89-4e89-a3b5-846a8373a569 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:09.371447 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 390/779] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:33:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:33:09.376652 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f352c1-3e5b-46a1-a1ae-ead67cdf3ba6 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:09.376940 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f352c1-3e5b-46a1-a1ae-ead67cdf3ba6 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:09.377214 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f352c1-3e5b-46a1-a1ae-ead67cdf3ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:09.396047 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-c1f352c1-3e5b-46a1-a1ae-ead67cdf3ba6 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:33:09.397209 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-c1f352c1-3e5b-46a1-a1ae-ead67cdf3ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:33:09.473885 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 390/780] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:33:09 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 08:33:09.646419 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af3184-8a3c-4de3-832d-c8729b09a571 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:09.646708 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af3184-8a3c-4de3-832d-c8729b09a571 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:09.646896 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af3184-8a3c-4de3-832d-c8729b09a571 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:09.648498 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 391/781] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:33:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:33:09.653300 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66304914-efd7-4b82-b4c7-768ba67fabc6 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:09.653589 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66304914-efd7-4b82-b4c7-768ba67fabc6 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:09.653895 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66304914-efd7-4b82-b4c7-768ba67fabc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:09.672304 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-66304914-efd7-4b82-b4c7-768ba67fabc6 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:33:09.673481 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-66304914-efd7-4b82-b4c7-768ba67fabc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:33:09.720452 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:09.742261 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85af3184-8a3c-4de3-832d-c8729b09a571 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:09.746758 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 391/782] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:33:09 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 08:33:09.805971 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:09.845143 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85af3184-8a3c-4de3-832d-c8729b09a571 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:09.878049 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e401ac-d227-4457-b7f2-d8b4c262ad1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:09.878673 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e401ac-d227-4457-b7f2-d8b4c262ad1d None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:09.878957 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e401ac-d227-4457-b7f2-d8b4c262ad1d None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:09.879228 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e401ac-d227-4457-b7f2-d8b4c262ad1d None admin] PATH_INFO: `/v3/services/image` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:09.879800 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e401ac-d227-4457-b7f2-d8b4c262ad1d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:09.881233 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e401ac-d227-4457-b7f2-d8b4c262ad1d None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:09.889859 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-90e401ac-d227-4457-b7f2-d8b4c262ad1d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 15 08:33:09.889859 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:33:09.889859 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 08:33:09.889859 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 08:33:09.889859 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 08:33:09.889859 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 08:33:09.889859 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 08:33:09.889859 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 08:33:09.889859 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 15 08:33:09.889859 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:33:09.889859 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 08:33:09.889859 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:33:09.889859 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:33:09.889859 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:33:09.889859 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:33:09.889859 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:33:09.889859 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:33:09.890927 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:33:09.890927 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:33:09.890927 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:33:09.890927 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:33:09.890927 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:33:09.890927 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:33:09.890927 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 08:33:09.890927 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 08:33:09.890927 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 08:33:09.890927 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 08:33:09.890927 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:33:09.890927 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:33:09.890927 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:33:09.890927 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:33:09.890927 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:33:09.890927 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:33:09.894788 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:33:09.894788 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:33:09.894788 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:33:09.894788 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:33:09.894788 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:33:09.894788 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:33:09.894788 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:33:09.894788 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:33:09.894788 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:33:09.894788 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:33:09.894788 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 08:33:09.894788 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 08:33:09.894788 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 15 08:33:09.894788 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:33:09.894788 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 392/783] 10.222.0.30 () {56 vars in 1105 bytes} [Sat Oct 15 08:33:09 2022] GET /identity/v3/services/image => generated 86 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 08:33:09.897445 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66304914-efd7-4b82-b4c7-768ba67fabc6 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:09.999062 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66304914-efd7-4b82-b4c7-768ba67fabc6 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:10.016777 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e401ac-d227-4457-b7f2-d8b4c262ad1d None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:10.034195 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e088a553-30eb-4683-9ee4-cdd355ac8d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:10.034947 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e088a553-30eb-4683-9ee4-cdd355ac8d28 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:10.035230 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e088a553-30eb-4683-9ee4-cdd355ac8d28 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:10.035502 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e088a553-30eb-4683-9ee4-cdd355ac8d28 None admin] PATH_INFO: `/v3/services` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:10.036120 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e088a553-30eb-4683-9ee4-cdd355ac8d28 None admin] RBAC: Adding query filter params (name=image) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:33:10.036423 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e088a553-30eb-4683-9ee4-cdd355ac8d28 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:10.037817 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e088a553-30eb-4683-9ee4-cdd355ac8d28 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:10.043087 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 392/784] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:33:09 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:33:10.048705 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e088a553-30eb-4683-9ee4-cdd355ac8d28 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:10.095663 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:10.131583 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e401ac-d227-4457-b7f2-d8b4c262ad1d None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:10.155889 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e088a553-30eb-4683-9ee4-cdd355ac8d28 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:10.159227 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:10.169604 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ebcca3-d06d-4b87-b608-cdebb6c28aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:10.170368 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ebcca3-d06d-4b87-b608-cdebb6c28aa8 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:10.170647 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ebcca3-d06d-4b87-b608-cdebb6c28aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:10.170917 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ebcca3-d06d-4b87-b608-cdebb6c28aa8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:10.181883 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ebcca3-d06d-4b87-b608-cdebb6c28aa8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:33:10.186906 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-82ebcca3-d06d-4b87-b608-cdebb6c28aa8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 15 08:33:10.186906 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:33:10.186906 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:33:10.186906 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:33:10.186906 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:33:10.186906 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:33:10.186906 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:33:10.186906 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:33:10.186906 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:33:10.186906 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:33:10.186906 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:33:10.186906 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:33:10.186906 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:33:10.186906 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:33:10.186906 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:33:10.186906 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:33:10.187838 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:33:10.187838 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:33:10.187838 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:33:10.187838 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:33:10.187838 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:33:10.187838 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:33:10.187838 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:33:10.187838 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:33:10.187838 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:33:10.187838 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:33:10.187838 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:33:10.187838 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:33:10.187838 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:33:10.187838 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:33:10.187838 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:33:10.187838 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:33:10.189420 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:33:10.189420 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:33:10.189420 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:33:10.189420 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:33:10.189420 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:33:10.189420 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:33:10.189420 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:33:10.189420 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:33:10.189420 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:33:10.189420 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:33:10.189420 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 15 08:33:10.189420 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:33:10.189420 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 393/785] 10.222.0.30 () {56 vars in 1120 bytes} [Sat Oct 15 08:33:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 08:33:10.191744 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb376843-ea96-4887-93bd-df2e14334ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:10.192354 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb376843-ea96-4887-93bd-df2e14334ba2 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:10.192647 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb376843-ea96-4887-93bd-df2e14334ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:10.192912 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb376843-ea96-4887-93bd-df2e14334ba2 None admin] PATH_INFO: `/v3/services` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:10.193536 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb376843-ea96-4887-93bd-df2e14334ba2 None admin] RBAC: Adding query filter params (type=image) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:33:10.193879 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb376843-ea96-4887-93bd-df2e14334ba2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:10.193974 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ebcca3-d06d-4b87-b608-cdebb6c28aa8 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:10.195812 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb376843-ea96-4887-93bd-df2e14334ba2 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:10.205416 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 393/786] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:33:10 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 08:33:10.206160 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb376843-ea96-4887-93bd-df2e14334ba2 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:10.299174 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ebcca3-d06d-4b87-b608-cdebb6c28aa8 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:10.308355 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb376843-ea96-4887-93bd-df2e14334ba2 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:10.316646 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:10.334346 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eed469-9b3d-4a8f-b791-6990f67b927f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:10.334946 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eed469-9b3d-4a8f-b791-6990f67b927f None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:10.335254 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eed469-9b3d-4a8f-b791-6990f67b927f None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:10.335544 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eed469-9b3d-4a8f-b791-6990f67b927f None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:10.336203 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eed469-9b3d-4a8f-b791-6990f67b927f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:33:10.336506 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eed469-9b3d-4a8f-b791-6990f67b927f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:10.337876 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eed469-9b3d-4a8f-b791-6990f67b927f None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:10.350530 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 394/787] 10.222.0.30 () {56 vars in 1115 bytes} [Sat Oct 15 08:33:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:33:10.350696 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e60cae-e81a-4574-ad43-636f27d039a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:10.350696 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e60cae-e81a-4574-ad43-636f27d039a2 None admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:10.350696 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e60cae-e81a-4574-ad43-636f27d039a2 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:10.350696 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e60cae-e81a-4574-ad43-636f27d039a2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:10.351597 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e60cae-e81a-4574-ad43-636f27d039a2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:10.351597 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e60cae-e81a-4574-ad43-636f27d039a2 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:10.352205 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eed469-9b3d-4a8f-b791-6990f67b927f None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:10.369951 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 394/788] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:33:10 2022] POST /identity/v3/endpoints => generated 327 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 08:33:10.411803 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:10.464057 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eed469-9b3d-4a8f-b791-6990f67b927f None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:10.506779 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1ef5e8-a6ce-4653-9abe-6e12eee2c77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:10.507505 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1ef5e8-a6ce-4653-9abe-6e12eee2c77b None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:10.507788 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1ef5e8-a6ce-4653-9abe-6e12eee2c77b None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:10.508059 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1ef5e8-a6ce-4653-9abe-6e12eee2c77b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:10.519602 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1ef5e8-a6ce-4653-9abe-6e12eee2c77b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:10.521084 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1ef5e8-a6ce-4653-9abe-6e12eee2c77b None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:10.537514 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-5e1ef5e8-a6ce-4653-9abe-6e12eee2c77b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:33:10.537514 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:33:10.537514 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:33:10.537514 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:33:10.537514 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:33:10.537514 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:33:10.537514 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:33:10.537514 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:33:10.537514 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:33:10.537514 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:33:10.537514 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:33:10.537514 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:33:10.537514 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 08:33:10.537514 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 08:33:10.537514 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 08:33:10.537514 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 08:33:10.538770 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:33:10.538770 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:33:10.538770 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:33:10.538770 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:33:10.538770 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:33:10.538770 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:33:10.538770 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:33:10.538770 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:33:10.538770 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:33:10.538770 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:33:10.538770 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:33:10.538770 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:33:10.538770 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:33:10.538770 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:33:10.538770 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:33:10.538770 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:33:10.538770 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 08:33:10.539689 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 08:33:10.539689 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:33:10.539689 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:33:10.542032 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 395/789] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:33:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:33:10.544540 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e60cae-e81a-4574-ad43-636f27d039a2 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:10.651773 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e60cae-e81a-4574-ad43-636f27d039a2 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:10.685386 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4ff5c4-f1f0-41dc-84d4-2474b0331350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:10.685947 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ff5c4-f1f0-41dc-84d4-2474b0331350 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:10.686229 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ff5c4-f1f0-41dc-84d4-2474b0331350 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:10.686502 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ff5c4-f1f0-41dc-84d4-2474b0331350 None admin] PATH_INFO: `/v3/domains` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:10.687148 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4ff5c4-f1f0-41dc-84d4-2474b0331350 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:33:10.687453 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4ff5c4-f1f0-41dc-84d4-2474b0331350 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:10.688842 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4ff5c4-f1f0-41dc-84d4-2474b0331350 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:10.700982 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 395/790] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 08:33:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:33:10.706734 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1ef5e8-a6ce-4653-9abe-6e12eee2c77b None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:10.805106 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:10.812282 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1ef5e8-a6ce-4653-9abe-6e12eee2c77b None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:10.846689 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806f5bd8-b1cd-4787-82a1-f03612c0ef93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:10.847445 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f5bd8-b1cd-4787-82a1-f03612c0ef93 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:10.847801 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f5bd8-b1cd-4787-82a1-f03612c0ef93 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:10.848152 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f5bd8-b1cd-4787-82a1-f03612c0ef93 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:10.857648 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806f5bd8-b1cd-4787-82a1-f03612c0ef93 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:10.859260 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806f5bd8-b1cd-4787-82a1-f03612c0ef93 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:10.872487 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-806f5bd8-b1cd-4787-82a1-f03612c0ef93 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 15 08:33:10.872487 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:33:10.872487 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:33:10.872487 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:33:10.872487 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:33:10.872487 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:33:10.872487 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:33:10.872487 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:33:10.872487 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:33:10.872487 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:33:10.872487 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:33:10.872487 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:33:10.872487 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 08:33:10.872487 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 08:33:10.872487 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 08:33:10.872487 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 08:33:10.873834 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:33:10.873834 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:33:10.873834 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 08:33:10.873834 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 08:33:10.873834 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 08:33:10.873834 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 08:33:10.873834 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:33:10.873834 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:33:10.873834 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:33:10.873834 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:33:10.873834 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:33:10.873834 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:33:10.873834 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:33:10.873834 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:33:10.873834 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:33:10.873834 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:33:10.873834 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:33:10.873834 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:33:10.875041 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:33:10.875041 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:33:10.875041 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 08:33:10.875041 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 08:33:10.875041 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 08:33:10.875041 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 08:33:10.875041 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 08:33:10.875041 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 08:33:10.875041 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 15 08:33:10.875041 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:33:10.875861 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 396/791] 10.222.0.30 () {56 vars in 1093 bytes} [Sat Oct 15 08:33:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:33:10.881106 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4ff5c4-f1f0-41dc-84d4-2474b0331350 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:10.907973 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:10.963604 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a6b70-6416-416e-843c-7a0e676582da None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:10.963802 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a6b70-6416-416e-843c-7a0e676582da None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:10.964079 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a6b70-6416-416e-843c-7a0e676582da None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:10.965709 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 397/792] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:33:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 08:33:10.970481 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca8400d-fea9-4210-b92c-51db43fb1874 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:10.970838 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca8400d-fea9-4210-b92c-51db43fb1874 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:10.971191 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca8400d-fea9-4210-b92c-51db43fb1874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:10.990777 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-2ca8400d-fea9-4210-b92c-51db43fb1874 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:33:10.992465 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-2ca8400d-fea9-4210-b92c-51db43fb1874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:33:10.993289 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4ff5c4-f1f0-41dc-84d4-2474b0331350 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:11.029537 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c306caee-f40c-4de8-919b-d92239cc3fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:11.030273 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306caee-f40c-4de8-919b-d92239cc3fbd None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:11.030623 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306caee-f40c-4de8-919b-d92239cc3fbd None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:11.030967 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306caee-f40c-4de8-919b-d92239cc3fbd None admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:11.031760 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c306caee-f40c-4de8-919b-d92239cc3fbd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:33:11.032126 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c306caee-f40c-4de8-919b-d92239cc3fbd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:11.033610 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c306caee-f40c-4de8-919b-d92239cc3fbd None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:11.055011 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 396/793] 10.222.0.30 () {56 vars in 1133 bytes} [Sat Oct 15 08:33:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:33:11.060432 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c306caee-f40c-4de8-919b-d92239cc3fbd None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:11.069066 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 398/794] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:33:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 08:33:11.142697 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:11.163677 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c306caee-f40c-4de8-919b-d92239cc3fbd None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:11.197191 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caad285-c025-4da4-8d7c-646c405fe162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:11.197982 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caad285-c025-4da4-8d7c-646c405fe162 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:11.198359 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caad285-c025-4da4-8d7c-646c405fe162 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:11.198727 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caad285-c025-4da4-8d7c-646c405fe162 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:11.208700 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caad285-c025-4da4-8d7c-646c405fe162 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:11.210270 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caad285-c025-4da4-8d7c-646c405fe162 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:11.225108 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-1caad285-c025-4da4-8d7c-646c405fe162 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:33:11.225108 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:33:11.225108 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:33:11.225108 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:33:11.225108 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:33:11.225108 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:33:11.225108 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:33:11.225108 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:33:11.225108 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:33:11.225108 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:33:11.225108 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:33:11.225108 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:33:11.225108 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 08:33:11.225108 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 08:33:11.225108 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 08:33:11.225108 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 08:33:11.226282 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:33:11.226282 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:33:11.226282 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:33:11.226282 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:33:11.226282 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:33:11.226282 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:33:11.226282 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:33:11.226282 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:33:11.226282 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:33:11.226282 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:33:11.226282 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:33:11.226282 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:33:11.226282 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:33:11.226282 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:33:11.226282 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:33:11.226282 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:33:11.226282 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 08:33:11.227393 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 08:33:11.227393 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:33:11.227393 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:33:11.228527 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 397/795] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:33:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 08:33:11.233995 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca8400d-fea9-4210-b92c-51db43fb1874 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:11.318763 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:11.335033 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca8400d-fea9-4210-b92c-51db43fb1874 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:11.339253 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caad285-c025-4da4-8d7c-646c405fe162 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:11.370419 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9243fe-9113-4de8-a75e-4203e69e84db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:11.371100 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9243fe-9113-4de8-a75e-4203e69e84db None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:11.371502 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9243fe-9113-4de8-a75e-4203e69e84db None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:11.371721 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9243fe-9113-4de8-a75e-4203e69e84db None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:11.372357 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9243fe-9113-4de8-a75e-4203e69e84db None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:33:11.372657 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9243fe-9113-4de8-a75e-4203e69e84db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:11.374068 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9243fe-9113-4de8-a75e-4203e69e84db None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:11.395127 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 399/796] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 08:33:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:33:11.396243 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9243fe-9113-4de8-a75e-4203e69e84db None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:11.454411 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caad285-c025-4da4-8d7c-646c405fe162 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:11.485759 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:11.492406 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec2c14d-e07b-4b66-b440-1a4aaf6b66a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:11.493011 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec2c14d-e07b-4b66-b440-1a4aaf6b66a3 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:11.493714 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec2c14d-e07b-4b66-b440-1a4aaf6b66a3 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:11.493965 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec2c14d-e07b-4b66-b440-1a4aaf6b66a3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:11.495054 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec2c14d-e07b-4b66-b440-1a4aaf6b66a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:11.496843 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec2c14d-e07b-4b66-b440-1a4aaf6b66a3 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:11.498604 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 398/797] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:33:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:33:11.504014 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec2c14d-e07b-4b66-b440-1a4aaf6b66a3 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:11.505857 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9243fe-9113-4de8-a75e-4203e69e84db None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:11.545489 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443acd65-b125-49b2-86d4-2a50da9132c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:11.546142 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443acd65-b125-49b2-86d4-2a50da9132c7 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:11.546420 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443acd65-b125-49b2-86d4-2a50da9132c7 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:11.546697 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443acd65-b125-49b2-86d4-2a50da9132c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:11.553464 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443acd65-b125-49b2-86d4-2a50da9132c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:11.554960 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443acd65-b125-49b2-86d4-2a50da9132c7 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:11.566001 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-443acd65-b125-49b2-86d4-2a50da9132c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 08:33:11.566001 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:33:11.566001 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:33:11.566001 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:33:11.566001 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:33:11.566001 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:33:11.566001 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:33:11.566001 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:33:11.566001 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:33:11.566001 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:33:11.566001 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:33:11.566001 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:33:11.566001 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 08:33:11.566001 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 08:33:11.566001 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 08:33:11.566911 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 08:33:11.566911 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:33:11.566911 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:33:11.566911 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:33:11.566911 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:33:11.566911 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:33:11.566911 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:33:11.566911 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:33:11.566911 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:33:11.566911 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:33:11.566911 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:33:11.566911 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:33:11.566911 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:33:11.566911 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:33:11.566911 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:33:11.566911 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:33:11.566911 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:33:11.567828 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 08:33:11.567828 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 08:33:11.567828 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 08:33:11.567828 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 08:33:11.567828 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 08:33:11.567828 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 08:33:11.567828 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 08:33:11.567828 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:33:11.567828 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 400/798] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 08:33:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:33:11.573583 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443acd65-b125-49b2-86d4-2a50da9132c7 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:11.585598 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:11.615248 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec2c14d-e07b-4b66-b440-1a4aaf6b66a3 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:11.651251 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7972aef6-b10a-4180-9506-72d64d6c23f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:11.651869 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7972aef6-b10a-4180-9506-72d64d6c23f8 None admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:11.652152 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7972aef6-b10a-4180-9506-72d64d6c23f8 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:11.652427 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7972aef6-b10a-4180-9506-72d64d6c23f8 None admin] PATH_INFO: `/v3/projects` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:11.653074 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7972aef6-b10a-4180-9506-72d64d6c23f8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:11.654539 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7972aef6-b10a-4180-9506-72d64d6c23f8 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:11.684590 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 399/799] 10.222.0.30 () {60 vars in 1141 bytes} [Sat Oct 15 08:33:11 2022] POST /identity/v3/projects => generated 311 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 08:33:11.686366 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443acd65-b125-49b2-86d4-2a50da9132c7 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:11.724897 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b6927c-26c0-46e2-92b7-a624a6f86e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:11.725518 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b6927c-26c0-46e2-92b7-a624a6f86e7b None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:11.725849 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b6927c-26c0-46e2-92b7-a624a6f86e7b None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:11.726671 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b6927c-26c0-46e2-92b7-a624a6f86e7b None admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:11.727372 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b6927c-26c0-46e2-92b7-a624a6f86e7b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:33:11.727679 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b6927c-26c0-46e2-92b7-a624a6f86e7b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:11.729112 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b6927c-26c0-46e2-92b7-a624a6f86e7b None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:11.743615 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 401/800] 10.222.0.30 () {56 vars in 1148 bytes} [Sat Oct 15 08:33:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 08:33:11.749438 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7972aef6-b10a-4180-9506-72d64d6c23f8 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:11.809148 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:11.864071 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7972aef6-b10a-4180-9506-72d64d6c23f8 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:11.901575 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910656bc-58ff-4ff8-9b8e-4fb79cc22869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:11.902261 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910656bc-58ff-4ff8-9b8e-4fb79cc22869 None admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:11.902541 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910656bc-58ff-4ff8-9b8e-4fb79cc22869 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:11.902814 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910656bc-58ff-4ff8-9b8e-4fb79cc22869 None admin] PATH_INFO: `/v3/projects/5dfceb59b3794439a4586b3cbe408f0e/users/af0a8af5cadc4cc1ab73e8eb2a20bb7e/roles/3e2674a06b1e46739ff0be6a5e20003e` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:11.904938 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910656bc-58ff-4ff8-9b8e-4fb79cc22869 None admin] RBAC: Authorizing `identity:create_grant(project_id=5dfceb59b3794439a4586b3cbe408f0e, user_id=af0a8af5cadc4cc1ab73e8eb2a20bb7e, role_id=3e2674a06b1e46739ff0be6a5e20003e)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:11.906415 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910656bc-58ff-4ff8-9b8e-4fb79cc22869 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:11.920010 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 400/801] 10.222.0.30 () {58 vars in 1439 bytes} [Sat Oct 15 08:33:11 2022] PUT /identity/v3/projects/5dfceb59b3794439a4586b3cbe408f0e/users/af0a8af5cadc4cc1ab73e8eb2a20bb7e/roles/3e2674a06b1e46739ff0be6a5e20003e => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 08:33:12.585436 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a7bb07-0185-4511-bf97-6998bccc5f36 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:12.586000 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a7bb07-0185-4511-bf97-6998bccc5f36 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:12.586079 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a7bb07-0185-4511-bf97-6998bccc5f36 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:12.587519 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 402/802] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:33:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:33:12.592648 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691a3430-5e11-489d-a8e6-3cda34f524e1 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:12.592937 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691a3430-5e11-489d-a8e6-3cda34f524e1 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:12.593213 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691a3430-5e11-489d-a8e6-3cda34f524e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:12.613212 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-691a3430-5e11-489d-a8e6-3cda34f524e1 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:33:12.614460 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-691a3430-5e11-489d-a8e6-3cda34f524e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:33:12.691512 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 401/803] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:33:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 08:33:12.957280 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a7bb07-0185-4511-bf97-6998bccc5f36 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:13.024328 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:13.061356 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a7bb07-0185-4511-bf97-6998bccc5f36 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:13.094211 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e0d5d7-445d-421c-a860-976a090f1f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:13.094818 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e0d5d7-445d-421c-a860-976a090f1f0d None admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:13.095110 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e0d5d7-445d-421c-a860-976a090f1f0d None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:13.095378 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e0d5d7-445d-421c-a860-976a090f1f0d None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:13.096050 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e0d5d7-445d-421c-a860-976a090f1f0d None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:13.097402 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e0d5d7-445d-421c-a860-976a090f1f0d None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:13.105687 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.sql.core [None req-20e0d5d7-445d-421c-a860-976a090f1f0d None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 15 08:33:13.105687 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 15 08:33:13.105687 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [parameters: {'id': 'ccbbf77fa5304b6dbaeadb7ecd329ce4', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 15 08:33:13.105687 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88059) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 15 08:33:13.113800 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-20e0d5d7-445d-421c-a860-976a090f1f0d None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 15 08:33:13.113800 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:33:13.113800 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 15 08:33:13.113800 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 15 08:33:13.113800 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 15 08:33:13.113800 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 15 08:33:13.113800 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 15 08:33:13.113800 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application result = self._query(query) Oct 15 08:33:13.113800 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 15 08:33:13.113800 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application conn.query(q) Oct 15 08:33:13.113800 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 15 08:33:13.113800 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 15 08:33:13.113800 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 15 08:33:13.113800 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application result.read() Oct 15 08:33:13.114651 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 15 08:33:13.114651 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 15 08:33:13.114651 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 15 08:33:13.114651 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 15 08:33:13.114651 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 15 08:33:13.114651 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 15 08:33:13.114651 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 15 08:33:13.114651 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 15 08:33:13.114651 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 15 08:33:13.114651 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:33:13.114651 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 15 08:33:13.114651 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:33:13.114651 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:33:13.114651 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 15 08:33:13.114651 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 15 08:33:13.114651 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 15 08:33:13.114651 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 15 08:33:13.114651 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 15 08:33:13.115671 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application next(self.gen) Oct 15 08:33:13.115671 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 15 08:33:13.115671 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application yield resource Oct 15 08:33:13.115671 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 15 08:33:13.115671 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application next(self.gen) Oct 15 08:33:13.115671 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 15 08:33:13.115671 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self.session.rollback() Oct 15 08:33:13.115671 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 08:33:13.115671 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self.force_reraise() Oct 15 08:33:13.115671 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 08:33:13.115671 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise self.value Oct 15 08:33:13.115671 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 15 08:33:13.115671 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 15 08:33:13.115671 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 15 08:33:13.115671 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application session.commit() Oct 15 08:33:13.115671 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 15 08:33:13.115671 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 15 08:33:13.116627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 15 08:33:13.116627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._prepare_impl() Oct 15 08:33:13.116627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 15 08:33:13.116627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self.session.flush() Oct 15 08:33:13.116627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 15 08:33:13.116627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._flush(objects) Oct 15 08:33:13.116627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 15 08:33:13.116627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 15 08:33:13.116627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 15 08:33:13.116627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application compat.raise_( Oct 15 08:33:13.116627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 15 08:33:13.116627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception Oct 15 08:33:13.116627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 15 08:33:13.116627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application flush_context.execute() Oct 15 08:33:13.116627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 15 08:33:13.116627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rec.execute(self) Oct 15 08:33:13.116627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 15 08:33:13.117524 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 15 08:33:13.117524 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 15 08:33:13.117524 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 15 08:33:13.117524 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 15 08:33:13.117524 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 15 08:33:13.117524 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 15 08:33:13.117524 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 15 08:33:13.117524 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 15 08:33:13.117524 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 15 08:33:13.117524 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 15 08:33:13.117524 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 15 08:33:13.117524 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 15 08:33:13.117524 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 15 08:33:13.117524 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 15 08:33:13.117524 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 15 08:33:13.118395 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 15 08:33:13.118395 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception Oct 15 08:33:13.118395 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 15 08:33:13.118395 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 15 08:33:13.118395 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 15 08:33:13.118395 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 15 08:33:13.118395 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 15 08:33:13.118395 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application result = self._query(query) Oct 15 08:33:13.118395 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 15 08:33:13.118395 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application conn.query(q) Oct 15 08:33:13.118395 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 15 08:33:13.118395 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 15 08:33:13.118395 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 15 08:33:13.118395 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application result.read() Oct 15 08:33:13.118395 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 15 08:33:13.118395 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 15 08:33:13.118395 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 15 08:33:13.119271 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 15 08:33:13.119271 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 15 08:33:13.119271 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 15 08:33:13.119271 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 15 08:33:13.119271 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 15 08:33:13.119271 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 15 08:33:13.119271 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 15 08:33:13.119271 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [parameters: {'id': 'ccbbf77fa5304b6dbaeadb7ecd329ce4', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 15 08:33:13.119271 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 15 08:33:13.119271 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:33:13.119271 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 08:33:13.119271 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:33:13.119271 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:33:13.119271 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 15 08:33:13.119271 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 15 08:33:13.120051 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 15 08:33:13.120051 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 15 08:33:13.120051 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 15 08:33:13.120051 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:33:13.120051 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 08:33:13.120051 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:33:13.120051 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:33:13.120051 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:33:13.120051 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:33:13.120051 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:33:13.120051 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:33:13.120051 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:33:13.120051 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:33:13.120051 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:33:13.120051 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:33:13.120051 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:33:13.120051 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:33:13.120976 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 15 08:33:13.120976 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 15 08:33:13.120976 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:33:13.120976 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:33:13.120976 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 15 08:33:13.120976 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 15 08:33:13.120976 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:33:13.120976 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:33:13.120976 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 15 08:33:13.120976 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 15 08:33:13.120976 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 15 08:33:13.120976 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:33:13.120976 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 403/804] 10.222.0.30 () {60 vars in 1137 bytes} [Sat Oct 15 08:33:12 2022] POST /identity/v3/domains => generated 190 bytes in 158 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:33:13.124460 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691a3430-5e11-489d-a8e6-3cda34f524e1 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:13.172281 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:13.230488 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691a3430-5e11-489d-a8e6-3cda34f524e1 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:13.263997 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64a83e5-e3e8-417a-9d4e-6f50b29e1873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:13.264627 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64a83e5-e3e8-417a-9d4e-6f50b29e1873 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:13.264964 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64a83e5-e3e8-417a-9d4e-6f50b29e1873 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:13.265241 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64a83e5-e3e8-417a-9d4e-6f50b29e1873 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:13.275343 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64a83e5-e3e8-417a-9d4e-6f50b29e1873 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:13.276797 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64a83e5-e3e8-417a-9d4e-6f50b29e1873 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:13.291643 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-d64a83e5-e3e8-417a-9d4e-6f50b29e1873 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:33:13.291643 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:33:13.291643 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:33:13.291643 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:33:13.291643 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:33:13.291643 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:33:13.291643 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:33:13.291643 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:33:13.291643 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:33:13.291643 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:33:13.291643 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:33:13.291643 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:33:13.291643 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 08:33:13.291643 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 08:33:13.291643 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 08:33:13.291643 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 08:33:13.292652 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:33:13.292652 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:33:13.292652 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:33:13.292652 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:33:13.292652 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:33:13.292652 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:33:13.292652 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:33:13.292652 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:33:13.292652 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:33:13.292652 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:33:13.292652 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:33:13.292652 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:33:13.292652 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:33:13.292652 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:33:13.292652 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:33:13.292652 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:33:13.292652 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 08:33:13.293575 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 08:33:13.293575 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:33:13.293575 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:33:13.293575 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 402/805] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:33:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:33:13.300429 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e0d5d7-445d-421c-a860-976a090f1f0d None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:13.414317 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e0d5d7-445d-421c-a860-976a090f1f0d None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:13.443049 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:13.450371 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bbbdf5-747a-44c1-b4ce-cfefb5d91d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:13.450987 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bbbdf5-747a-44c1-b4ce-cfefb5d91d7c None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:13.451301 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bbbdf5-747a-44c1-b4ce-cfefb5d91d7c None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:13.451617 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bbbdf5-747a-44c1-b4ce-cfefb5d91d7c None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:13.452296 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bbbdf5-747a-44c1-b4ce-cfefb5d91d7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:33:13.452958 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bbbdf5-747a-44c1-b4ce-cfefb5d91d7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:13.454899 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bbbdf5-747a-44c1-b4ce-cfefb5d91d7c None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:13.467400 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 404/806] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 08:33:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:33:13.903849 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5909c69-fc18-4141-8135-5ce186b97e68 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:13.904420 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5909c69-fc18-4141-8135-5ce186b97e68 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:13.904420 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5909c69-fc18-4141-8135-5ce186b97e68 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:13.905851 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 403/807] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:33:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:33:13.911208 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a9f868-4533-492c-bfd1-1db94a66e167 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:13.911620 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a9f868-4533-492c-bfd1-1db94a66e167 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:13.911764 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a9f868-4533-492c-bfd1-1db94a66e167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:13.929448 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-95a9f868-4533-492c-bfd1-1db94a66e167 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:33:13.930664 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-95a9f868-4533-492c-bfd1-1db94a66e167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:33:13.999118 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 405/808] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:33:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 08:33:14.059033 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef40a55-b825-4484-a74b-705b6e6d3359 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:14.059320 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef40a55-b825-4484-a74b-705b6e6d3359 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:14.059596 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef40a55-b825-4484-a74b-705b6e6d3359 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:14.060940 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 404/809] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:33:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:33:14.066168 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07af551-760c-4e39-8a74-2bf739bd21b7 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:14.066456 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07af551-760c-4e39-8a74-2bf739bd21b7 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:14.066732 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07af551-760c-4e39-8a74-2bf739bd21b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:14.084267 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-f07af551-760c-4e39-8a74-2bf739bd21b7 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:33:14.085468 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-f07af551-760c-4e39-8a74-2bf739bd21b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:33:14.149585 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 406/810] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:33:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 08:33:14.270032 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef40a55-b825-4484-a74b-705b6e6d3359 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:14.361853 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:14.370316 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef40a55-b825-4484-a74b-705b6e6d3359 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:14.406818 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c43bd2-1c70-48ae-8263-d2322557743c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:14.407457 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c43bd2-1c70-48ae-8263-d2322557743c None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:14.407741 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c43bd2-1c70-48ae-8263-d2322557743c None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:14.408017 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c43bd2-1c70-48ae-8263-d2322557743c None admin] PATH_INFO: `/v3/domains/default` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:14.409128 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c43bd2-1c70-48ae-8263-d2322557743c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:14.410593 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c43bd2-1c70-48ae-8263-d2322557743c None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:14.412166 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 405/811] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:33:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:33:14.418403 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07af551-760c-4e39-8a74-2bf739bd21b7 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:14.421048 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c43bd2-1c70-48ae-8263-d2322557743c None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:14.469584 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:14.519540 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07af551-760c-4e39-8a74-2bf739bd21b7 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:14.520677 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c43bd2-1c70-48ae-8263-d2322557743c None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:14.551972 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a238c1ae-8e2e-4bc6-86e8-2e76c2539559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:14.552419 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb2b255-2258-4c8f-9f12-0e8ef257b54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:14.552561 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a238c1ae-8e2e-4bc6-86e8-2e76c2539559 None admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:14.552849 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a238c1ae-8e2e-4bc6-86e8-2e76c2539559 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:14.553038 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb2b255-2258-4c8f-9f12-0e8ef257b54d None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:14.553137 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a238c1ae-8e2e-4bc6-86e8-2e76c2539559 None admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:14.553319 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb2b255-2258-4c8f-9f12-0e8ef257b54d None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:14.553589 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb2b255-2258-4c8f-9f12-0e8ef257b54d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:14.553814 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a238c1ae-8e2e-4bc6-86e8-2e76c2539559 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:14.555778 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a238c1ae-8e2e-4bc6-86e8-2e76c2539559 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:14.564554 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb2b255-2258-4c8f-9f12-0e8ef257b54d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:33:14.569421 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-eeb2b255-2258-4c8f-9f12-0e8ef257b54d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 15 08:33:14.569421 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:33:14.569421 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:33:14.569421 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:33:14.569421 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:33:14.569421 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:33:14.569421 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:33:14.569421 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:33:14.569421 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:33:14.569421 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:33:14.569421 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:33:14.569421 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:33:14.569421 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:33:14.569421 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:33:14.569421 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:33:14.569421 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:33:14.576689 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:33:14.576689 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:33:14.576689 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:33:14.576689 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:33:14.576689 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:33:14.576689 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:33:14.576689 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:33:14.576689 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:33:14.576689 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:33:14.576689 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:33:14.576689 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:33:14.576689 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:33:14.576689 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:33:14.576689 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:33:14.576689 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:33:14.576689 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:33:14.577698 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:33:14.577698 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:33:14.577698 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:33:14.577698 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:33:14.577698 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:33:14.577698 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:33:14.577698 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:33:14.577698 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:33:14.577698 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:33:14.577698 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:33:14.577698 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 15 08:33:14.577698 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:33:14.577698 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 406/812] 10.222.0.30 () {56 vars in 1120 bytes} [Sat Oct 15 08:33:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 08:33:14.577698 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb2b255-2258-4c8f-9f12-0e8ef257b54d None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:14.579206 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 407/813] 10.222.0.30 () {60 vars in 1132 bytes} [Sat Oct 15 08:33:14 2022] POST /identity/v3/users => generated 285 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:33:14.604259 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:14.687062 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb2b255-2258-4c8f-9f12-0e8ef257b54d None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:14.722519 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f6c3bb-d3de-4188-8c58-5826ddb5670c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:14.723137 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6c3bb-d3de-4188-8c58-5826ddb5670c None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:14.723414 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6c3bb-d3de-4188-8c58-5826ddb5670c None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:14.723688 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6c3bb-d3de-4188-8c58-5826ddb5670c None admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:14.724358 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f6c3bb-d3de-4188-8c58-5826ddb5670c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:33:14.724659 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f6c3bb-d3de-4188-8c58-5826ddb5670c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:14.726761 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f6c3bb-d3de-4188-8c58-5826ddb5670c None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:14.735514 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 407/814] 10.222.0.30 () {56 vars in 1115 bytes} [Sat Oct 15 08:33:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 08:33:14.741406 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a238c1ae-8e2e-4bc6-86e8-2e76c2539559 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:14.839957 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a238c1ae-8e2e-4bc6-86e8-2e76c2539559 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:14.847887 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:14.876001 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11dddcf-b4e8-4434-a182-c564dcb4d3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:14.876629 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11dddcf-b4e8-4434-a182-c564dcb4d3a1 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:14.876912 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11dddcf-b4e8-4434-a182-c564dcb4d3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:14.877185 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11dddcf-b4e8-4434-a182-c564dcb4d3a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:14.886894 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11dddcf-b4e8-4434-a182-c564dcb4d3a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:14.888320 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11dddcf-b4e8-4434-a182-c564dcb4d3a1 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:14.902734 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-a11dddcf-b4e8-4434-a182-c564dcb4d3a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:33:14.902734 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:33:14.902734 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:33:14.902734 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:33:14.902734 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:33:14.902734 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:33:14.902734 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:33:14.902734 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:33:14.902734 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:33:14.902734 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:33:14.902734 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:33:14.902734 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:33:14.902734 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 08:33:14.902734 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 08:33:14.902734 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 08:33:14.902734 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 08:33:14.903904 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:33:14.903904 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:33:14.903904 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:33:14.903904 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:33:14.903904 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:33:14.903904 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:33:14.903904 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:33:14.903904 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:33:14.903904 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:33:14.903904 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:33:14.903904 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:33:14.903904 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:33:14.903904 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:33:14.903904 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:33:14.903904 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:33:14.903904 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:33:14.903904 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 08:33:14.905079 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 08:33:14.905079 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:33:14.905079 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:33:14.905823 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 408/815] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:33:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 08:33:14.909458 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f6c3bb-d3de-4188-8c58-5826ddb5670c None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:14.994204 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:15.010014 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f6c3bb-d3de-4188-8c58-5826ddb5670c None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:15.044745 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945caabc-8a85-4553-bd42-88461e6d24fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:15.045443 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945caabc-8a85-4553-bd42-88461e6d24fd None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:15.045909 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945caabc-8a85-4553-bd42-88461e6d24fd None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:15.046216 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945caabc-8a85-4553-bd42-88461e6d24fd None admin] PATH_INFO: `/v3/domains` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:15.047409 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945caabc-8a85-4553-bd42-88461e6d24fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:33:15.047794 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945caabc-8a85-4553-bd42-88461e6d24fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:15.049168 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945caabc-8a85-4553-bd42-88461e6d24fd None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:15.061670 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 408/816] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 08:33:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:33:15.067124 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11dddcf-b4e8-4434-a182-c564dcb4d3a1 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:15.165485 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11dddcf-b4e8-4434-a182-c564dcb4d3a1 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:15.197565 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e71ca0-555b-490a-8cd7-1f48b4e7d98a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:15.198213 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e71ca0-555b-490a-8cd7-1f48b4e7d98a None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:15.198495 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e71ca0-555b-490a-8cd7-1f48b4e7d98a None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:15.198772 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e71ca0-555b-490a-8cd7-1f48b4e7d98a None admin] PATH_INFO: `/v3/users/nova` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:15.205932 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e71ca0-555b-490a-8cd7-1f48b4e7d98a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:15.207357 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e71ca0-555b-490a-8cd7-1f48b4e7d98a None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:15.218708 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-e9e71ca0-555b-490a-8cd7-1f48b4e7d98a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 15 08:33:15.218708 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:33:15.218708 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:33:15.218708 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:33:15.218708 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:33:15.218708 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:33:15.218708 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:33:15.218708 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:33:15.218708 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:33:15.218708 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:33:15.218708 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:33:15.218708 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:33:15.218708 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 08:33:15.218708 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 08:33:15.218708 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 08:33:15.218708 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 08:33:15.219665 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:33:15.219665 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:33:15.219665 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 08:33:15.219665 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 08:33:15.219665 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 08:33:15.219665 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 08:33:15.219665 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:33:15.219665 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:33:15.219665 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:33:15.219665 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:33:15.219665 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:33:15.219665 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:33:15.219665 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:33:15.219665 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:33:15.219665 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:33:15.219665 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:33:15.219665 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:33:15.219665 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:33:15.220659 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:33:15.220659 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:33:15.220659 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 08:33:15.220659 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 08:33:15.220659 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 08:33:15.220659 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 08:33:15.220659 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 08:33:15.220659 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 08:33:15.220659 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 15 08:33:15.220659 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:33:15.222056 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 409/817] 10.222.0.30 () {56 vars in 1093 bytes} [Sat Oct 15 08:33:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:33:15.225546 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945caabc-8a85-4553-bd42-88461e6d24fd None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:15.324312 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945caabc-8a85-4553-bd42-88461e6d24fd None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:15.357850 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9080e273-6555-4d49-aa2a-4b0a2c3f4053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:15.358456 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9080e273-6555-4d49-aa2a-4b0a2c3f4053 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:15.358732 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9080e273-6555-4d49-aa2a-4b0a2c3f4053 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:15.359001 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9080e273-6555-4d49-aa2a-4b0a2c3f4053 None admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:15.359719 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9080e273-6555-4d49-aa2a-4b0a2c3f4053 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:33:15.360025 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9080e273-6555-4d49-aa2a-4b0a2c3f4053 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:15.361447 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9080e273-6555-4d49-aa2a-4b0a2c3f4053 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:15.378010 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 409/818] 10.222.0.30 () {56 vars in 1133 bytes} [Sat Oct 15 08:33:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:33:15.384299 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e71ca0-555b-490a-8cd7-1f48b4e7d98a None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:15.405040 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:15.483823 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e71ca0-555b-490a-8cd7-1f48b4e7d98a None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:15.517694 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7704e9ec-b9ec-40fe-8d01-6afdbf260907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:15.518306 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7704e9ec-b9ec-40fe-8d01-6afdbf260907 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:15.518589 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7704e9ec-b9ec-40fe-8d01-6afdbf260907 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:15.518862 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7704e9ec-b9ec-40fe-8d01-6afdbf260907 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:15.582792 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7704e9ec-b9ec-40fe-8d01-6afdbf260907 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:15.584225 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7704e9ec-b9ec-40fe-8d01-6afdbf260907 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:15.602035 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:15.679887 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 410/819] 10.222.0.30 () {58 vars in 1282 bytes} [Sat Oct 15 08:33:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 15 08:33:15.685723 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9080e273-6555-4d49-aa2a-4b0a2c3f4053 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:15.688597 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:15.785565 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9080e273-6555-4d49-aa2a-4b0a2c3f4053 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:15.818736 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f020f2-87d4-4b56-80d3-4389299c86ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:15.819369 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f020f2-87d4-4b56-80d3-4389299c86ad None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:15.819686 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f020f2-87d4-4b56-80d3-4389299c86ad None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:15.819961 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f020f2-87d4-4b56-80d3-4389299c86ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:15.829720 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f020f2-87d4-4b56-80d3-4389299c86ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:15.831161 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f020f2-87d4-4b56-80d3-4389299c86ad None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:15.845891 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-02f020f2-87d4-4b56-80d3-4389299c86ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:33:15.845891 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:33:15.845891 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:33:15.845891 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:33:15.845891 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:33:15.845891 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:33:15.845891 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:33:15.845891 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:33:15.845891 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:33:15.845891 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:33:15.845891 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:33:15.845891 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:33:15.845891 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 08:33:15.845891 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 08:33:15.845891 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 08:33:15.845891 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 08:33:15.846995 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:33:15.846995 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:33:15.846995 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:33:15.846995 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:33:15.846995 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:33:15.846995 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:33:15.846995 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:33:15.846995 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:33:15.846995 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:33:15.846995 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:33:15.846995 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:33:15.846995 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:33:15.846995 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:33:15.846995 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:33:15.846995 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:33:15.846995 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:33:15.846995 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 08:33:15.847866 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 08:33:15.847866 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:33:15.847866 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:33:15.847866 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 410/820] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:33:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:33:15.852651 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7704e9ec-b9ec-40fe-8d01-6afdbf260907 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:15.948535 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7704e9ec-b9ec-40fe-8d01-6afdbf260907 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:15.965117 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:15.981556 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a55ba1-1ba7-4506-b8e8-7ae7503d40b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:15.982183 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a55ba1-1ba7-4506-b8e8-7ae7503d40b8 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:15.982463 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a55ba1-1ba7-4506-b8e8-7ae7503d40b8 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:15.982735 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a55ba1-1ba7-4506-b8e8-7ae7503d40b8 None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:15.983366 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a55ba1-1ba7-4506-b8e8-7ae7503d40b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:33:15.983666 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a55ba1-1ba7-4506-b8e8-7ae7503d40b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:15.985019 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a55ba1-1ba7-4506-b8e8-7ae7503d40b8 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:15.996249 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 411/821] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 08:33:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:33:16.001867 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f020f2-87d4-4b56-80d3-4389299c86ad None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:16.058128 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:16.102095 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f020f2-87d4-4b56-80d3-4389299c86ad None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:16.134632 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c2f0b0-996f-4f61-8b04-47e16335b846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:16.135251 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c2f0b0-996f-4f61-8b04-47e16335b846 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:16.135532 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c2f0b0-996f-4f61-8b04-47e16335b846 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:16.135805 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c2f0b0-996f-4f61-8b04-47e16335b846 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:16.142628 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c2f0b0-996f-4f61-8b04-47e16335b846 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:16.144081 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c2f0b0-996f-4f61-8b04-47e16335b846 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:16.155229 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-c3c2f0b0-996f-4f61-8b04-47e16335b846 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 08:33:16.155229 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:33:16.155229 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:33:16.155229 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:33:16.155229 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:33:16.155229 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:33:16.155229 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:33:16.155229 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:33:16.155229 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:33:16.155229 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:33:16.155229 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:33:16.155229 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:33:16.155229 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 08:33:16.155229 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 08:33:16.155229 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 08:33:16.156353 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 08:33:16.156353 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:33:16.156353 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:33:16.156353 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:33:16.156353 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:33:16.156353 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:33:16.156353 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:33:16.156353 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:33:16.156353 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:33:16.156353 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:33:16.156353 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:33:16.156353 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:33:16.156353 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:33:16.156353 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:33:16.156353 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:33:16.156353 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:33:16.156353 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:33:16.157275 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 08:33:16.157275 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 08:33:16.157275 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 08:33:16.157275 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 08:33:16.157275 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 08:33:16.157275 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 08:33:16.157275 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 08:33:16.157275 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:33:16.157976 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 411/822] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 08:33:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 08:33:16.162131 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a55ba1-1ba7-4506-b8e8-7ae7503d40b8 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:16.258811 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a55ba1-1ba7-4506-b8e8-7ae7503d40b8 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:16.290883 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a9998a-cec9-41e7-a41d-99b5046e1fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:16.291479 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a9998a-cec9-41e7-a41d-99b5046e1fea None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:16.291757 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a9998a-cec9-41e7-a41d-99b5046e1fea None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:16.292031 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a9998a-cec9-41e7-a41d-99b5046e1fea None admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:16.292704 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a9998a-cec9-41e7-a41d-99b5046e1fea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:33:16.293008 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a9998a-cec9-41e7-a41d-99b5046e1fea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:16.294447 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a9998a-cec9-41e7-a41d-99b5046e1fea None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:16.306462 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 412/823] 10.222.0.30 () {56 vars in 1148 bytes} [Sat Oct 15 08:33:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:33:16.312510 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c2f0b0-996f-4f61-8b04-47e16335b846 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:16.412853 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c2f0b0-996f-4f61-8b04-47e16335b846 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:16.426518 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:16.446933 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f35e556-6fac-48ff-a367-2bbbee61ccd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:16.447532 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f35e556-6fac-48ff-a367-2bbbee61ccd3 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:16.447812 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f35e556-6fac-48ff-a367-2bbbee61ccd3 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:16.448088 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f35e556-6fac-48ff-a367-2bbbee61ccd3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:16.448802 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f35e556-6fac-48ff-a367-2bbbee61ccd3 None admin] RBAC: Adding query filter params (role.id=3e2674a06b1e46739ff0be6a5e20003e, scope.project.id=5dfceb59b3794439a4586b3cbe408f0e, user.id=af0a8af5cadc4cc1ab73e8eb2a20bb7e) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:33:16.449111 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f35e556-6fac-48ff-a367-2bbbee61ccd3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:16.450564 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f35e556-6fac-48ff-a367-2bbbee61ccd3 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:16.456095 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 412/824] 10.222.0.30 () {56 vars in 1374 bytes} [Sat Oct 15 08:33:16 2022] GET /identity/v3/role_assignments?user.id=af0a8af5cadc4cc1ab73e8eb2a20bb7e&scope.project.id=5dfceb59b3794439a4586b3cbe408f0e&role.id=3e2674a06b1e46739ff0be6a5e20003e => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 08:33:16.755848 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf97c8-cedd-4f72-b2cd-60190311adac None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:16.756156 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf97c8-cedd-4f72-b2cd-60190311adac None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:16.756441 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf97c8-cedd-4f72-b2cd-60190311adac None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:16.757913 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 413/825] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:33:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:33:16.763337 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181096be-cb6b-4d9f-9b24-205150c08ff0 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:16.763618 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181096be-cb6b-4d9f-9b24-205150c08ff0 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:16.763896 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181096be-cb6b-4d9f-9b24-205150c08ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:16.783409 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-181096be-cb6b-4d9f-9b24-205150c08ff0 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:33:16.784622 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-181096be-cb6b-4d9f-9b24-205150c08ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:33:16.852364 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 413/826] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:33:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 08:33:17.117603 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bf97c8-cedd-4f72-b2cd-60190311adac None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:17.159401 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:17.220474 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bf97c8-cedd-4f72-b2cd-60190311adac None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:17.256655 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc3b850-d3ce-4a57-9eaf-31caa616ae71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:17.257918 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc3b850-d3ce-4a57-9eaf-31caa616ae71 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:17.258203 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc3b850-d3ce-4a57-9eaf-31caa616ae71 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:17.258478 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc3b850-d3ce-4a57-9eaf-31caa616ae71 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:17.268671 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc3b850-d3ce-4a57-9eaf-31caa616ae71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:33:17.273469 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-2dc3b850-d3ce-4a57-9eaf-31caa616ae71 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 08:33:17.273469 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:33:17.273469 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:33:17.273469 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:33:17.273469 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:33:17.273469 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:33:17.273469 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:33:17.273469 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:33:17.273469 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:33:17.273469 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:33:17.273469 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:33:17.273469 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:33:17.273469 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:33:17.273469 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:33:17.273469 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:33:17.273469 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:33:17.274484 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:33:17.274484 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:33:17.274484 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:33:17.274484 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:33:17.274484 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:33:17.274484 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:33:17.274484 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:33:17.274484 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:33:17.274484 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:33:17.274484 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:33:17.274484 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:33:17.274484 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:33:17.274484 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:33:17.274484 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:33:17.274484 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:33:17.274484 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:33:17.275348 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:33:17.275348 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:33:17.275348 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:33:17.275348 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:33:17.275348 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:33:17.275348 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:33:17.275348 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:33:17.275348 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:33:17.275348 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:33:17.275348 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:33:17.275348 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 08:33:17.275348 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:33:17.275348 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 414/827] 10.222.0.30 () {56 vars in 1096 bytes} [Sat Oct 15 08:33:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:33:17.280714 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181096be-cb6b-4d9f-9b24-205150c08ff0 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:17.375854 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181096be-cb6b-4d9f-9b24-205150c08ff0 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:17.385232 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:17.410473 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c2eca3-5522-47ae-88d0-3ca866785d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:17.411077 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2eca3-5522-47ae-88d0-3ca866785d06 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:17.411355 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2eca3-5522-47ae-88d0-3ca866785d06 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:17.411632 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2eca3-5522-47ae-88d0-3ca866785d06 None admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:17.412307 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c2eca3-5522-47ae-88d0-3ca866785d06 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:33:17.412607 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c2eca3-5522-47ae-88d0-3ca866785d06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:17.414048 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c2eca3-5522-47ae-88d0-3ca866785d06 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:17.421763 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 414/828] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 08:33:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:33:17.427566 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc3b850-d3ce-4a57-9eaf-31caa616ae71 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:17.519573 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:17.523015 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc3b850-d3ce-4a57-9eaf-31caa616ae71 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:17.555483 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32345e53-9278-41e3-a03b-c285fb810906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:17.556088 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32345e53-9278-41e3-a03b-c285fb810906 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:17.556365 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32345e53-9278-41e3-a03b-c285fb810906 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:17.556664 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32345e53-9278-41e3-a03b-c285fb810906 None admin] PATH_INFO: `/v3/users/6d45d929eaf34f2e9727b86d056ee394` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:17.568978 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32345e53-9278-41e3-a03b-c285fb810906 None admin] RBAC: Authorizing `identity:get_user(user_id=6d45d929eaf34f2e9727b86d056ee394)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:17.570995 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32345e53-9278-41e3-a03b-c285fb810906 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:17.572549 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 415/829] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 08:33:17 2022] GET /identity/v3/users/6d45d929eaf34f2e9727b86d056ee394 => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:33:17.578131 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c2eca3-5522-47ae-88d0-3ca866785d06 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:17.674075 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c2eca3-5522-47ae-88d0-3ca866785d06 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:17.706143 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa132446-197b-45c0-a6dc-0dce808b50dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:17.706754 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa132446-197b-45c0-a6dc-0dce808b50dc None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:17.707035 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa132446-197b-45c0-a6dc-0dce808b50dc None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:17.707309 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa132446-197b-45c0-a6dc-0dce808b50dc None admin] PATH_INFO: `/v3/projects/1bb0415a16ed4df2a963c99364d8c0b7` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:17.708399 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa132446-197b-45c0-a6dc-0dce808b50dc None admin] RBAC: Authorizing `identity:get_project(project_id=1bb0415a16ed4df2a963c99364d8c0b7)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:17.709853 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa132446-197b-45c0-a6dc-0dce808b50dc None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:17.711542 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 415/830] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 08:33:17 2022] GET /identity/v3/projects/1bb0415a16ed4df2a963c99364d8c0b7 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:33:17.717175 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32345e53-9278-41e3-a03b-c285fb810906 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:17.811713 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32345e53-9278-41e3-a03b-c285fb810906 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:17.843080 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebf1d0d-5450-4ab0-a7ec-63b3c23717be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:17.843709 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf1d0d-5450-4ab0-a7ec-63b3c23717be None admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:17.843987 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf1d0d-5450-4ab0-a7ec-63b3c23717be None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:17.844261 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf1d0d-5450-4ab0-a7ec-63b3c23717be None admin] PATH_INFO: `/v3/projects/1bb0415a16ed4df2a963c99364d8c0b7/users/6d45d929eaf34f2e9727b86d056ee394/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:17.846121 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebf1d0d-5450-4ab0-a7ec-63b3c23717be None admin] RBAC: Authorizing `identity:create_grant(project_id=1bb0415a16ed4df2a963c99364d8c0b7, user_id=6d45d929eaf34f2e9727b86d056ee394, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:17.847513 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebf1d0d-5450-4ab0-a7ec-63b3c23717be None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:17.859888 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 416/831] 10.222.0.30 () {58 vars in 1439 bytes} [Sat Oct 15 08:33:17 2022] PUT /identity/v3/projects/1bb0415a16ed4df2a963c99364d8c0b7/users/6d45d929eaf34f2e9727b86d056ee394/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:33:19.941804 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a39374-ee6e-47a6-96da-c93a4853be5b None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:19.942328 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a39374-ee6e-47a6-96da-c93a4853be5b None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:19.942445 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a39374-ee6e-47a6-96da-c93a4853be5b None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:19.943720 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 416/832] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:33:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:33:19.948843 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7eddee-4826-4bf3-baf7-b5d31741bdf9 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:19.949277 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7eddee-4826-4bf3-baf7-b5d31741bdf9 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:19.949397 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7eddee-4826-4bf3-baf7-b5d31741bdf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:19.967099 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-5e7eddee-4826-4bf3-baf7-b5d31741bdf9 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:33:19.968284 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-5e7eddee-4826-4bf3-baf7-b5d31741bdf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:33:19.979061 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:20.034518 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 417/833] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:33:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 08:33:20.304087 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a39374-ee6e-47a6-96da-c93a4853be5b None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:20.343654 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:20.403350 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a39374-ee6e-47a6-96da-c93a4853be5b None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:20.435542 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1c0209-6584-4e4b-a852-cc975ae34ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:20.436159 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1c0209-6584-4e4b-a852-cc975ae34ecd None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:20.436446 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1c0209-6584-4e4b-a852-cc975ae34ecd None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:20.436712 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1c0209-6584-4e4b-a852-cc975ae34ecd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:20.446814 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1c0209-6584-4e4b-a852-cc975ae34ecd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:33:20.451525 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-ec1c0209-6584-4e4b-a852-cc975ae34ecd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 08:33:20.451525 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:33:20.451525 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:33:20.451525 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:33:20.451525 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:33:20.451525 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:33:20.451525 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:33:20.451525 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:33:20.451525 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:33:20.451525 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:33:20.451525 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:33:20.451525 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:33:20.451525 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:33:20.451525 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:33:20.451525 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:33:20.451525 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:33:20.452606 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:33:20.452606 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:33:20.452606 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:33:20.452606 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:33:20.452606 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:33:20.452606 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:33:20.452606 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:33:20.452606 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:33:20.452606 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:33:20.452606 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:33:20.452606 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:33:20.452606 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:33:20.452606 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:33:20.452606 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:33:20.452606 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:33:20.452606 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:33:20.453456 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:33:20.453456 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:33:20.453456 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:33:20.453456 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:33:20.453456 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:33:20.453456 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:33:20.453456 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:33:20.453456 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:33:20.453456 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:33:20.453456 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:33:20.453456 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 08:33:20.453456 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:33:20.453456 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 417/834] 10.222.0.30 () {56 vars in 1096 bytes} [Sat Oct 15 08:33:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:33:20.458758 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7eddee-4826-4bf3-baf7-b5d31741bdf9 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:20.554177 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7eddee-4826-4bf3-baf7-b5d31741bdf9 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:20.586432 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45c46ca-0311-4874-8fa9-fff5d074c2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:20.587025 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45c46ca-0311-4874-8fa9-fff5d074c2cf None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:20.587308 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45c46ca-0311-4874-8fa9-fff5d074c2cf None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:20.587593 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45c46ca-0311-4874-8fa9-fff5d074c2cf None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:20.588247 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45c46ca-0311-4874-8fa9-fff5d074c2cf None admin] RBAC: Adding query filter params (name=admin) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:33:20.588552 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45c46ca-0311-4874-8fa9-fff5d074c2cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:20.589941 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45c46ca-0311-4874-8fa9-fff5d074c2cf None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:20.597601 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 418/835] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 08:33:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:33:20.604146 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1c0209-6584-4e4b-a852-cc975ae34ecd None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:20.697551 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:20.701338 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1c0209-6584-4e4b-a852-cc975ae34ecd None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:20.733432 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1df7cc-7fcc-46de-bb40-9992e345efaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:20.734068 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1df7cc-7fcc-46de-bb40-9992e345efaa None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:20.734347 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1df7cc-7fcc-46de-bb40-9992e345efaa None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:20.734618 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1df7cc-7fcc-46de-bb40-9992e345efaa None admin] PATH_INFO: `/v3/users/6d45d929eaf34f2e9727b86d056ee394` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:20.735705 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1df7cc-7fcc-46de-bb40-9992e345efaa None admin] RBAC: Authorizing `identity:get_user(user_id=6d45d929eaf34f2e9727b86d056ee394)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:20.737126 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1df7cc-7fcc-46de-bb40-9992e345efaa None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:20.738693 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 418/836] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 08:33:20 2022] GET /identity/v3/users/6d45d929eaf34f2e9727b86d056ee394 => generated 285 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:33:20.744397 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45c46ca-0311-4874-8fa9-fff5d074c2cf None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:20.755471 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:20.841265 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45c46ca-0311-4874-8fa9-fff5d074c2cf None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:20.872602 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cae100a-60e1-4740-8912-5c738af883be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:20.873188 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae100a-60e1-4740-8912-5c738af883be None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:20.873471 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae100a-60e1-4740-8912-5c738af883be None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:20.873770 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae100a-60e1-4740-8912-5c738af883be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:20.937396 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cae100a-60e1-4740-8912-5c738af883be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:20.938851 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cae100a-60e1-4740-8912-5c738af883be None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:20.948360 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:21.032895 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 419/837] 10.222.0.30 () {58 vars in 1282 bytes} [Sat Oct 15 08:33:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 15 08:33:21.038745 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1df7cc-7fcc-46de-bb40-9992e345efaa None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:21.136745 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1df7cc-7fcc-46de-bb40-9992e345efaa None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:21.168464 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9485d216-aafb-42b3-a82b-b7559e450ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:21.169065 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485d216-aafb-42b3-a82b-b7559e450ded None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:21.169341 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485d216-aafb-42b3-a82b-b7559e450ded None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:21.169612 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485d216-aafb-42b3-a82b-b7559e450ded None admin] PATH_INFO: `/v3/projects/1bb0415a16ed4df2a963c99364d8c0b7` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:21.170722 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9485d216-aafb-42b3-a82b-b7559e450ded None admin] RBAC: Authorizing `identity:get_project(project_id=1bb0415a16ed4df2a963c99364d8c0b7)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:21.172147 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9485d216-aafb-42b3-a82b-b7559e450ded None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:21.173756 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 419/838] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 08:33:21 2022] GET /identity/v3/projects/1bb0415a16ed4df2a963c99364d8c0b7 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:33:21.180177 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cae100a-60e1-4740-8912-5c738af883be None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:21.274507 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cae100a-60e1-4740-8912-5c738af883be None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:21.283376 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:21.307984 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f33b64-3daa-446d-8f42-d15bfdc6ceee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:21.308581 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f33b64-3daa-446d-8f42-d15bfdc6ceee None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:21.308862 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f33b64-3daa-446d-8f42-d15bfdc6ceee None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:21.309137 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f33b64-3daa-446d-8f42-d15bfdc6ceee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:21.309885 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f33b64-3daa-446d-8f42-d15bfdc6ceee None admin] RBAC: Adding query filter params (role.id=14615e30ed3f4d8cb78ee5e0bf9b5761, scope.project.id=1bb0415a16ed4df2a963c99364d8c0b7, user.id=6d45d929eaf34f2e9727b86d056ee394) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:33:21.310198 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f33b64-3daa-446d-8f42-d15bfdc6ceee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:21.311611 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f33b64-3daa-446d-8f42-d15bfdc6ceee None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:21.317186 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 420/839] 10.222.0.30 () {56 vars in 1374 bytes} [Sat Oct 15 08:33:21 2022] GET /identity/v3/role_assignments?user.id=6d45d929eaf34f2e9727b86d056ee394&scope.project.id=1bb0415a16ed4df2a963c99364d8c0b7&role.id=14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:33:23.433470 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faa58a6-ae75-4435-b23f-2083c0df4e4f None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:23.433943 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faa58a6-ae75-4435-b23f-2083c0df4e4f None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:23.434057 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faa58a6-ae75-4435-b23f-2083c0df4e4f None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:23.435406 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 420/840] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:33:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:33:23.440764 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cfd32b-e575-45bb-9b99-c5312126963d None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:23.441198 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cfd32b-e575-45bb-9b99-c5312126963d None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:23.441316 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cfd32b-e575-45bb-9b99-c5312126963d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:23.458830 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-b8cfd32b-e575-45bb-9b99-c5312126963d None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:33:23.460507 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-b8cfd32b-e575-45bb-9b99-c5312126963d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:33:23.523468 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 421/841] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:33:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 08:33:23.788819 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faa58a6-ae75-4435-b23f-2083c0df4e4f None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:23.823521 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:23.887642 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faa58a6-ae75-4435-b23f-2083c0df4e4f None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:23.919825 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ed8939-ff90-481b-a6a4-a106a385c1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:23.920434 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed8939-ff90-481b-a6a4-a106a385c1d4 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:23.920711 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed8939-ff90-481b-a6a4-a106a385c1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:23.920982 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed8939-ff90-481b-a6a4-a106a385c1d4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:23.922093 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ed8939-ff90-481b-a6a4-a106a385c1d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:23.923512 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ed8939-ff90-481b-a6a4-a106a385c1d4 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:23.925042 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 421/842] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:33:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:33:23.930856 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cfd32b-e575-45bb-9b99-c5312126963d None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:24.025762 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cfd32b-e575-45bb-9b99-c5312126963d None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:24.027202 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:24.059317 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cba1cd-f172-4dff-80a6-8c2d8495d2b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:24.059849 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cba1cd-f172-4dff-80a6-8c2d8495d2b3 None admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:24.060129 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cba1cd-f172-4dff-80a6-8c2d8495d2b3 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:24.060401 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cba1cd-f172-4dff-80a6-8c2d8495d2b3 None admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:24.061031 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cba1cd-f172-4dff-80a6-8c2d8495d2b3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:24.062459 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cba1cd-f172-4dff-80a6-8c2d8495d2b3 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:24.082954 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 422/843] 10.222.0.30 () {60 vars in 1132 bytes} [Sat Oct 15 08:33:23 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:33:26.202769 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f3fc9-a2c8-4286-aa0d-4b1ebca55747 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:26.203334 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f3fc9-a2c8-4286-aa0d-4b1ebca55747 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:26.203334 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f3fc9-a2c8-4286-aa0d-4b1ebca55747 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:26.204642 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 422/844] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:33:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:33:26.209738 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad55ca-9788-46c6-b2d6-9b95ae8f3f25 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:26.210120 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad55ca-9788-46c6-b2d6-9b95ae8f3f25 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:26.210369 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad55ca-9788-46c6-b2d6-9b95ae8f3f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:26.228016 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-88ad55ca-9788-46c6-b2d6-9b95ae8f3f25 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:33:26.229198 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-88ad55ca-9788-46c6-b2d6-9b95ae8f3f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:33:26.292910 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 423/845] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:33:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 08:33:26.560168 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15f3fc9-a2c8-4286-aa0d-4b1ebca55747 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:26.657491 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15f3fc9-a2c8-4286-aa0d-4b1ebca55747 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:26.658985 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:26.691512 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3050a6-54f1-45e4-af96-9a21c2868f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:26.692120 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3050a6-54f1-45e4-af96-9a21c2868f33 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:26.692396 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3050a6-54f1-45e4-af96-9a21c2868f33 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:26.692666 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3050a6-54f1-45e4-af96-9a21c2868f33 None admin] PATH_INFO: `/v3/roles/d236b691c1434103b73cc4fd0b39fec1` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:26.694044 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3050a6-54f1-45e4-af96-9a21c2868f33 None admin] RBAC: Authorizing `identity:get_role(role_id=d236b691c1434103b73cc4fd0b39fec1)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:26.695431 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3050a6-54f1-45e4-af96-9a21c2868f33 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:26.696725 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 423/846] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 08:33:26 2022] GET /identity/v3/roles/d236b691c1434103b73cc4fd0b39fec1 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:33:26.702483 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ad55ca-9788-46c6-b2d6-9b95ae8f3f25 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:26.785067 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:26.797767 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ad55ca-9788-46c6-b2d6-9b95ae8f3f25 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:26.830761 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e701e3-2d3b-4e0a-ba27-f0494cb23538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:26.831327 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e701e3-2d3b-4e0a-ba27-f0494cb23538 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:26.831612 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e701e3-2d3b-4e0a-ba27-f0494cb23538 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:26.831893 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e701e3-2d3b-4e0a-ba27-f0494cb23538 None admin] PATH_INFO: `/v3/users/ea2e97fae6194bb482f87e1af8ebe274` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:26.844411 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e701e3-2d3b-4e0a-ba27-f0494cb23538 None admin] RBAC: Authorizing `identity:get_user(user_id=ea2e97fae6194bb482f87e1af8ebe274)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:26.845872 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e701e3-2d3b-4e0a-ba27-f0494cb23538 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:26.847449 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 424/847] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 08:33:26 2022] GET /identity/v3/users/ea2e97fae6194bb482f87e1af8ebe274 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:33:26.853615 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3050a6-54f1-45e4-af96-9a21c2868f33 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:26.949978 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3050a6-54f1-45e4-af96-9a21c2868f33 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:26.982198 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406765f1-9e4d-4806-b136-ac41ed4aa52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:26.982810 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406765f1-9e4d-4806-b136-ac41ed4aa52e None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:26.983094 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406765f1-9e4d-4806-b136-ac41ed4aa52e None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:26.983371 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406765f1-9e4d-4806-b136-ac41ed4aa52e None admin] PATH_INFO: `/v3/projects/1bb0415a16ed4df2a963c99364d8c0b7` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:26.984482 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406765f1-9e4d-4806-b136-ac41ed4aa52e None admin] RBAC: Authorizing `identity:get_project(project_id=1bb0415a16ed4df2a963c99364d8c0b7)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:26.985930 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406765f1-9e4d-4806-b136-ac41ed4aa52e None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:26.987530 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 424/848] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 08:33:26 2022] GET /identity/v3/projects/1bb0415a16ed4df2a963c99364d8c0b7 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:33:26.993714 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e701e3-2d3b-4e0a-ba27-f0494cb23538 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:27.089524 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e701e3-2d3b-4e0a-ba27-f0494cb23538 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:27.121779 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421f03cd-e345-4f8d-9b6b-a2e11caecd48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:27.122417 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f03cd-e345-4f8d-9b6b-a2e11caecd48 None admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:27.122696 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f03cd-e345-4f8d-9b6b-a2e11caecd48 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:27.122971 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f03cd-e345-4f8d-9b6b-a2e11caecd48 None admin] PATH_INFO: `/v3/projects/1bb0415a16ed4df2a963c99364d8c0b7/users/ea2e97fae6194bb482f87e1af8ebe274/roles/d236b691c1434103b73cc4fd0b39fec1` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:27.125061 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421f03cd-e345-4f8d-9b6b-a2e11caecd48 None admin] RBAC: Authorizing `identity:create_grant(project_id=1bb0415a16ed4df2a963c99364d8c0b7, user_id=ea2e97fae6194bb482f87e1af8ebe274, role_id=d236b691c1434103b73cc4fd0b39fec1)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:27.126495 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421f03cd-e345-4f8d-9b6b-a2e11caecd48 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:27.138823 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 425/849] 10.222.0.30 () {58 vars in 1439 bytes} [Sat Oct 15 08:33:26 2022] PUT /identity/v3/projects/1bb0415a16ed4df2a963c99364d8c0b7/users/ea2e97fae6194bb482f87e1af8ebe274/roles/d236b691c1434103b73cc4fd0b39fec1 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:33:29.250932 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d87321f-4be9-4052-be71-65aae7c5a994 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:29.251531 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d87321f-4be9-4052-be71-65aae7c5a994 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:29.251531 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d87321f-4be9-4052-be71-65aae7c5a994 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:29.252823 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 425/850] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:33:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:33:29.258790 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4393868-9bf4-4171-b12f-a91db8544208 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:29.259229 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4393868-9bf4-4171-b12f-a91db8544208 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:29.259358 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4393868-9bf4-4171-b12f-a91db8544208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:29.272447 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:29.277931 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-c4393868-9bf4-4171-b12f-a91db8544208 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:33:29.279295 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-c4393868-9bf4-4171-b12f-a91db8544208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:33:29.342952 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 426/851] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:33:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 08:33:29.612035 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d87321f-4be9-4052-be71-65aae7c5a994 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:29.652611 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:29.712387 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d87321f-4be9-4052-be71-65aae7c5a994 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:29.744735 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dae79de-901a-44a1-b852-9909aeb817be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:29.745350 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae79de-901a-44a1-b852-9909aeb817be None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:29.745634 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae79de-901a-44a1-b852-9909aeb817be None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:29.745941 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae79de-901a-44a1-b852-9909aeb817be None admin] PATH_INFO: `/v3/roles/d236b691c1434103b73cc4fd0b39fec1` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:29.747312 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dae79de-901a-44a1-b852-9909aeb817be None admin] RBAC: Authorizing `identity:get_role(role_id=d236b691c1434103b73cc4fd0b39fec1)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:29.748685 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dae79de-901a-44a1-b852-9909aeb817be None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:29.750014 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 426/852] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 08:33:29 2022] GET /identity/v3/roles/d236b691c1434103b73cc4fd0b39fec1 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:33:29.755630 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4393868-9bf4-4171-b12f-a91db8544208 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:29.849693 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4393868-9bf4-4171-b12f-a91db8544208 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:29.882334 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e476a14-712f-488b-98c6-97e3329da6e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:29.882887 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e476a14-712f-488b-98c6-97e3329da6e7 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:29.883165 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e476a14-712f-488b-98c6-97e3329da6e7 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:29.883435 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e476a14-712f-488b-98c6-97e3329da6e7 None admin] PATH_INFO: `/v3/users/ea2e97fae6194bb482f87e1af8ebe274` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:29.884537 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e476a14-712f-488b-98c6-97e3329da6e7 None admin] RBAC: Authorizing `identity:get_user(user_id=ea2e97fae6194bb482f87e1af8ebe274)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:29.885963 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e476a14-712f-488b-98c6-97e3329da6e7 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:29.887471 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 427/853] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 08:33:29 2022] GET /identity/v3/users/ea2e97fae6194bb482f87e1af8ebe274 => generated 286 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:33:29.893390 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dae79de-901a-44a1-b852-9909aeb817be None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:29.990550 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dae79de-901a-44a1-b852-9909aeb817be None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:29.997746 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:30.024965 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f795e766-6e28-4444-8059-94a8bec09061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:30.025561 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f795e766-6e28-4444-8059-94a8bec09061 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:30.025870 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f795e766-6e28-4444-8059-94a8bec09061 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:30.026148 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f795e766-6e28-4444-8059-94a8bec09061 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:30.091432 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f795e766-6e28-4444-8059-94a8bec09061 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:30.092875 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f795e766-6e28-4444-8059-94a8bec09061 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:30.187758 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 427/854] 10.222.0.30 () {58 vars in 1282 bytes} [Sat Oct 15 08:33:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 15 08:33:30.193526 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e476a14-712f-488b-98c6-97e3329da6e7 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:30.197842 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:30.288550 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e476a14-712f-488b-98c6-97e3329da6e7 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:30.320838 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e88c21-f2de-4919-9de2-6769bbce3930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:30.321457 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e88c21-f2de-4919-9de2-6769bbce3930 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:30.321770 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e88c21-f2de-4919-9de2-6769bbce3930 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:30.322050 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e88c21-f2de-4919-9de2-6769bbce3930 None admin] PATH_INFO: `/v3/projects/1bb0415a16ed4df2a963c99364d8c0b7` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:30.323160 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e88c21-f2de-4919-9de2-6769bbce3930 None admin] RBAC: Authorizing `identity:get_project(project_id=1bb0415a16ed4df2a963c99364d8c0b7)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:30.324578 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e88c21-f2de-4919-9de2-6769bbce3930 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:30.326219 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 428/855] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 08:33:30 2022] GET /identity/v3/projects/1bb0415a16ed4df2a963c99364d8c0b7 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:33:30.332369 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f795e766-6e28-4444-8059-94a8bec09061 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:30.340353 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:30.430586 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f795e766-6e28-4444-8059-94a8bec09061 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:30.462839 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0dd39d-0a48-46bb-8c58-46a0b16d62b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:30.463380 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0dd39d-0a48-46bb-8c58-46a0b16d62b4 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:30.463663 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0dd39d-0a48-46bb-8c58-46a0b16d62b4 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:30.463939 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0dd39d-0a48-46bb-8c58-46a0b16d62b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:30.464653 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0dd39d-0a48-46bb-8c58-46a0b16d62b4 None admin] RBAC: Adding query filter params (role.id=d236b691c1434103b73cc4fd0b39fec1, scope.project.id=1bb0415a16ed4df2a963c99364d8c0b7, user.id=ea2e97fae6194bb482f87e1af8ebe274) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:33:30.464966 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0dd39d-0a48-46bb-8c58-46a0b16d62b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:30.466421 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0dd39d-0a48-46bb-8c58-46a0b16d62b4 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:30.471960 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 428/856] 10.222.0.30 () {56 vars in 1374 bytes} [Sat Oct 15 08:33:30 2022] GET /identity/v3/role_assignments?user.id=ea2e97fae6194bb482f87e1af8ebe274&scope.project.id=1bb0415a16ed4df2a963c99364d8c0b7&role.id=d236b691c1434103b73cc4fd0b39fec1 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:33:32.571034 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250348a-335a-42e2-8656-f2af772a5ef7 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:32.571546 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250348a-335a-42e2-8656-f2af772a5ef7 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:32.571637 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250348a-335a-42e2-8656-f2af772a5ef7 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:32.572922 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 429/857] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:33:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:33:32.578149 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49f3d7-7e49-4481-9391-ebe56eac7374 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:32.578533 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49f3d7-7e49-4481-9391-ebe56eac7374 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:32.578691 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49f3d7-7e49-4481-9391-ebe56eac7374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:32.596361 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-ee49f3d7-7e49-4481-9391-ebe56eac7374 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:33:32.597529 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-ee49f3d7-7e49-4481-9391-ebe56eac7374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:33:32.657548 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:32.662067 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 429/858] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:33:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 08:33:32.924921 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3250348a-335a-42e2-8656-f2af772a5ef7 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:32.994202 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:33.024390 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3250348a-335a-42e2-8656-f2af772a5ef7 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:33.056229 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db10fa5-71d3-44b6-8b66-18fc5ebe9e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:33.056867 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db10fa5-71d3-44b6-8b66-18fc5ebe9e9d None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:33.057146 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db10fa5-71d3-44b6-8b66-18fc5ebe9e9d None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:33.057415 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db10fa5-71d3-44b6-8b66-18fc5ebe9e9d None admin] PATH_INFO: `/v3/domains/default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:33.058529 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db10fa5-71d3-44b6-8b66-18fc5ebe9e9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:33.059932 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db10fa5-71d3-44b6-8b66-18fc5ebe9e9d None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:33.061453 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 430/859] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:33:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:33:33.067169 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee49f3d7-7e49-4481-9391-ebe56eac7374 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:33.167017 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee49f3d7-7e49-4481-9391-ebe56eac7374 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:33.198505 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d356b95-8e49-4668-8007-a412ec828121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:33.199035 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d356b95-8e49-4668-8007-a412ec828121 None admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:33.199309 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d356b95-8e49-4668-8007-a412ec828121 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:33.199578 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d356b95-8e49-4668-8007-a412ec828121 None admin] PATH_INFO: `/v3/projects` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:33.200209 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d356b95-8e49-4668-8007-a412ec828121 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:33.201614 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d356b95-8e49-4668-8007-a412ec828121 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:33.227307 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 430/860] 10.222.0.30 () {60 vars in 1141 bytes} [Sat Oct 15 08:33:33 2022] POST /identity/v3/projects => generated 311 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:33:35.362001 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5a326d-adf4-44e0-a9b2-babb9525124c None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:35.362529 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5a326d-adf4-44e0-a9b2-babb9525124c None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:35.362617 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5a326d-adf4-44e0-a9b2-babb9525124c None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:35.363909 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 431/861] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:33:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:33:35.369135 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e2689-b99b-4fb0-a98b-a880ce56d87d None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:35.369536 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e2689-b99b-4fb0-a98b-a880ce56d87d None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:35.369702 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e2689-b99b-4fb0-a98b-a880ce56d87d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:35.387553 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-f59e2689-b99b-4fb0-a98b-a880ce56d87d None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:33:35.388729 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-f59e2689-b99b-4fb0-a98b-a880ce56d87d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:33:35.434912 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:35.454580 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 431/862] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:33:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 08:33:35.726588 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5a326d-adf4-44e0-a9b2-babb9525124c None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:35.825768 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5a326d-adf4-44e0-a9b2-babb9525124c None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:35.858030 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648884f9-aab9-463e-92a6-01ecb1c29b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:35.858643 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648884f9-aab9-463e-92a6-01ecb1c29b80 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:35.858927 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648884f9-aab9-463e-92a6-01ecb1c29b80 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:35.859206 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648884f9-aab9-463e-92a6-01ecb1c29b80 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:35.860307 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648884f9-aab9-463e-92a6-01ecb1c29b80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:35.861754 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648884f9-aab9-463e-92a6-01ecb1c29b80 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:35.863303 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 432/863] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:33:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:33:35.869018 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59e2689-b99b-4fb0-a98b-a880ce56d87d None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:35.963693 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59e2689-b99b-4fb0-a98b-a880ce56d87d None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:35.996473 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d09a30-b986-4a92-a91b-f86b403fdea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:35.997057 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d09a30-b986-4a92-a91b-f86b403fdea6 None admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:35.997333 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d09a30-b986-4a92-a91b-f86b403fdea6 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:35.997602 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d09a30-b986-4a92-a91b-f86b403fdea6 None admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:35.998262 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d09a30-b986-4a92-a91b-f86b403fdea6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:35.999673 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d09a30-b986-4a92-a91b-f86b403fdea6 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:36.020416 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 432/864] 10.222.0.30 () {60 vars in 1132 bytes} [Sat Oct 15 08:33:35 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:33:38.149421 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e494044-9d55-4929-bd76-889504fe613c None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:38.150011 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e494044-9d55-4929-bd76-889504fe613c None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:38.150011 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e494044-9d55-4929-bd76-889504fe613c None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:38.151336 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 433/865] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:33:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:33:38.156531 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff25b3-53e0-4f19-b41e-9b9b51d61dac None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:38.157296 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff25b3-53e0-4f19-b41e-9b9b51d61dac None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:38.157579 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff25b3-53e0-4f19-b41e-9b9b51d61dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:38.175453 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-26ff25b3-53e0-4f19-b41e-9b9b51d61dac None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:33:38.176641 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-26ff25b3-53e0-4f19-b41e-9b9b51d61dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:33:38.206271 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:38.244062 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 433/866] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:33:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 08:33:38.511491 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e494044-9d55-4929-bd76-889504fe613c None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:38.522635 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:38.611433 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e494044-9d55-4929-bd76-889504fe613c None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:38.644146 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27480bff-4460-448a-9e1c-b9ebcea3b25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:38.644763 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27480bff-4460-448a-9e1c-b9ebcea3b25d None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:38.645050 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27480bff-4460-448a-9e1c-b9ebcea3b25d None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:38.645327 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27480bff-4460-448a-9e1c-b9ebcea3b25d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:38.655342 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27480bff-4460-448a-9e1c-b9ebcea3b25d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:33:38.660170 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-27480bff-4460-448a-9e1c-b9ebcea3b25d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 08:33:38.660170 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:33:38.660170 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:33:38.660170 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:33:38.660170 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:33:38.660170 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:33:38.660170 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:33:38.660170 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:33:38.660170 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:33:38.660170 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:33:38.660170 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:33:38.660170 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:33:38.660170 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:33:38.660170 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:33:38.660170 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:33:38.660170 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:33:38.661344 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:33:38.661344 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:33:38.661344 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:33:38.661344 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:33:38.661344 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:33:38.661344 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:33:38.661344 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:33:38.661344 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:33:38.661344 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:33:38.661344 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:33:38.661344 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:33:38.661344 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:33:38.661344 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:33:38.661344 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:33:38.661344 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:33:38.661344 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:33:38.662627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:33:38.662627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:33:38.662627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:33:38.662627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:33:38.662627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:33:38.662627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:33:38.662627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:33:38.662627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:33:38.662627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:33:38.662627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:33:38.662627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 08:33:38.662627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:33:38.662627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 434/867] 10.222.0.30 () {56 vars in 1096 bytes} [Sat Oct 15 08:33:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:33:38.668336 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff25b3-53e0-4f19-b41e-9b9b51d61dac None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:38.764494 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff25b3-53e0-4f19-b41e-9b9b51d61dac None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:38.797309 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4119594-c0b3-4211-8d5e-f9bb91505d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:38.797922 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4119594-c0b3-4211-8d5e-f9bb91505d8c None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:38.798203 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4119594-c0b3-4211-8d5e-f9bb91505d8c None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:38.798475 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4119594-c0b3-4211-8d5e-f9bb91505d8c None admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:38.799125 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4119594-c0b3-4211-8d5e-f9bb91505d8c None admin] RBAC: Adding query filter params (name=admin) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:33:38.799427 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4119594-c0b3-4211-8d5e-f9bb91505d8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:38.800789 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4119594-c0b3-4211-8d5e-f9bb91505d8c None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:38.808593 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 434/868] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 08:33:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:33:38.814213 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27480bff-4460-448a-9e1c-b9ebcea3b25d None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:38.877835 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:38.909895 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27480bff-4460-448a-9e1c-b9ebcea3b25d None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:38.943315 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfbc10b-432f-4c46-9eab-ffd3259cf6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:38.943922 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfbc10b-432f-4c46-9eab-ffd3259cf6c8 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:38.944205 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfbc10b-432f-4c46-9eab-ffd3259cf6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:38.944480 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfbc10b-432f-4c46-9eab-ffd3259cf6c8 None admin] PATH_INFO: `/v3/users/e6c2d2245c0842b58721d49a52c2af92` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:38.956754 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfbc10b-432f-4c46-9eab-ffd3259cf6c8 None admin] RBAC: Authorizing `identity:get_user(user_id=e6c2d2245c0842b58721d49a52c2af92)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:38.958204 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfbc10b-432f-4c46-9eab-ffd3259cf6c8 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:38.960272 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 435/869] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 08:33:38 2022] GET /identity/v3/users/e6c2d2245c0842b58721d49a52c2af92 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:33:38.965828 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4119594-c0b3-4211-8d5e-f9bb91505d8c None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:38.996196 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:39.063136 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4119594-c0b3-4211-8d5e-f9bb91505d8c None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:39.094711 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e43f982-e7d1-4096-b8cc-96f31e083f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:39.095321 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e43f982-e7d1-4096-b8cc-96f31e083f73 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:39.095597 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e43f982-e7d1-4096-b8cc-96f31e083f73 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:39.095873 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e43f982-e7d1-4096-b8cc-96f31e083f73 None admin] PATH_INFO: `/v3/projects/9c9599f863ec41148d52358393690c9c` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:39.096954 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e43f982-e7d1-4096-b8cc-96f31e083f73 None admin] RBAC: Authorizing `identity:get_project(project_id=9c9599f863ec41148d52358393690c9c)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:39.098402 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e43f982-e7d1-4096-b8cc-96f31e083f73 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:39.099991 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 435/870] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 08:33:38 2022] GET /identity/v3/projects/9c9599f863ec41148d52358393690c9c => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:33:39.105893 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfbc10b-432f-4c46-9eab-ffd3259cf6c8 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:39.202108 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfbc10b-432f-4c46-9eab-ffd3259cf6c8 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:39.225806 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:39.234317 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fe0d2d-5d2e-4e9c-a0de-434660687571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:39.234957 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe0d2d-5d2e-4e9c-a0de-434660687571 None admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:39.235237 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe0d2d-5d2e-4e9c-a0de-434660687571 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:39.235511 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe0d2d-5d2e-4e9c-a0de-434660687571 None admin] PATH_INFO: `/v3/projects/9c9599f863ec41148d52358393690c9c/users/e6c2d2245c0842b58721d49a52c2af92/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:39.237346 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fe0d2d-5d2e-4e9c-a0de-434660687571 None admin] RBAC: Authorizing `identity:create_grant(project_id=9c9599f863ec41148d52358393690c9c, user_id=e6c2d2245c0842b58721d49a52c2af92, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:39.238782 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fe0d2d-5d2e-4e9c-a0de-434660687571 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:39.251082 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 436/871] 10.222.0.30 () {58 vars in 1439 bytes} [Sat Oct 15 08:33:39 2022] PUT /identity/v3/projects/9c9599f863ec41148d52358393690c9c/users/e6c2d2245c0842b58721d49a52c2af92/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:33:41.358780 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b06a905-5dff-4245-bd31-47a7159e8a2a None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:41.359309 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b06a905-5dff-4245-bd31-47a7159e8a2a None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:41.359309 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b06a905-5dff-4245-bd31-47a7159e8a2a None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:41.360670 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 436/872] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:33:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:33:41.365901 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbd7c58-853b-486f-9144-2e9b635ce115 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:41.366343 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbd7c58-853b-486f-9144-2e9b635ce115 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:41.366500 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbd7c58-853b-486f-9144-2e9b635ce115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:41.384150 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-2dbd7c58-853b-486f-9144-2e9b635ce115 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:33:41.385349 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-2dbd7c58-853b-486f-9144-2e9b635ce115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:33:41.449653 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 437/873] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:33:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 08:33:41.714911 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b06a905-5dff-4245-bd31-47a7159e8a2a None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:41.810025 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:41.815124 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b06a905-5dff-4245-bd31-47a7159e8a2a None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:41.846932 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce7122f-b022-4779-b065-1df984f0e19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:41.847549 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce7122f-b022-4779-b065-1df984f0e19a None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:41.847832 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce7122f-b022-4779-b065-1df984f0e19a None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:41.848107 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce7122f-b022-4779-b065-1df984f0e19a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:41.858906 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce7122f-b022-4779-b065-1df984f0e19a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:33:41.863628 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-cce7122f-b022-4779-b065-1df984f0e19a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 08:33:41.863628 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:33:41.863628 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:33:41.863628 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:33:41.863628 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:33:41.863628 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:33:41.863628 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:33:41.863628 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:33:41.863628 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:33:41.863628 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:33:41.863628 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:33:41.863628 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:33:41.863628 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:33:41.863628 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:33:41.863628 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:33:41.863628 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:33:41.864744 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:33:41.864744 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:33:41.864744 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:33:41.864744 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:33:41.864744 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:33:41.864744 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:33:41.864744 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:33:41.864744 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:33:41.864744 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:33:41.864744 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:33:41.864744 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:33:41.864744 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:33:41.864744 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:33:41.864744 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:33:41.864744 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:33:41.864744 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:33:41.865606 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:33:41.865606 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:33:41.865606 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:33:41.865606 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:33:41.865606 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:33:41.865606 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:33:41.865606 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:33:41.865606 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:33:41.865606 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:33:41.865606 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:33:41.865606 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 08:33:41.865606 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:33:41.865606 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 437/874] 10.222.0.30 () {56 vars in 1096 bytes} [Sat Oct 15 08:33:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:33:41.871130 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbd7c58-853b-486f-9144-2e9b635ce115 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:41.964878 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbd7c58-853b-486f-9144-2e9b635ce115 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:41.980881 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:41.998339 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc12ca37-62ec-45ea-925f-473706828a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:41.998935 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc12ca37-62ec-45ea-925f-473706828a9b None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:41.999211 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc12ca37-62ec-45ea-925f-473706828a9b None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:41.999481 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc12ca37-62ec-45ea-925f-473706828a9b None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:42.000133 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc12ca37-62ec-45ea-925f-473706828a9b None admin] RBAC: Adding query filter params (name=admin) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:33:42.000435 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc12ca37-62ec-45ea-925f-473706828a9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:42.001801 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc12ca37-62ec-45ea-925f-473706828a9b None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:42.009393 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 438/875] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 08:33:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:33:42.015083 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce7122f-b022-4779-b065-1df984f0e19a None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:42.112667 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce7122f-b022-4779-b065-1df984f0e19a None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:42.145367 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf55c7b-781b-4821-83f4-f914c3241750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:42.146007 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf55c7b-781b-4821-83f4-f914c3241750 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:42.146282 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf55c7b-781b-4821-83f4-f914c3241750 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:42.146551 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf55c7b-781b-4821-83f4-f914c3241750 None admin] PATH_INFO: `/v3/users/e6c2d2245c0842b58721d49a52c2af92` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:42.147660 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf55c7b-781b-4821-83f4-f914c3241750 None admin] RBAC: Authorizing `identity:get_user(user_id=e6c2d2245c0842b58721d49a52c2af92)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:42.149082 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf55c7b-781b-4821-83f4-f914c3241750 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:42.150646 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 438/876] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 08:33:42 2022] GET /identity/v3/users/e6c2d2245c0842b58721d49a52c2af92 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:33:42.156303 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc12ca37-62ec-45ea-925f-473706828a9b None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:42.251863 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc12ca37-62ec-45ea-925f-473706828a9b None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:42.285122 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c2050f-e99a-427a-bf9e-89593bd4c672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:42.285736 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2050f-e99a-427a-bf9e-89593bd4c672 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:42.286024 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2050f-e99a-427a-bf9e-89593bd4c672 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:42.286300 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2050f-e99a-427a-bf9e-89593bd4c672 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:42.329870 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:42.351242 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c2050f-e99a-427a-bf9e-89593bd4c672 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:42.352671 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c2050f-e99a-427a-bf9e-89593bd4c672 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:42.447454 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 439/877] 10.222.0.30 () {58 vars in 1282 bytes} [Sat Oct 15 08:33:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 15 08:33:42.453316 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf55c7b-781b-4821-83f4-f914c3241750 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:42.481305 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:42.551350 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf55c7b-781b-4821-83f4-f914c3241750 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:42.583534 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ffadb7-2a35-48a4-a574-e54b0ab73fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:42.584144 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ffadb7-2a35-48a4-a574-e54b0ab73fc9 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:42.584424 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ffadb7-2a35-48a4-a574-e54b0ab73fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:42.584845 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ffadb7-2a35-48a4-a574-e54b0ab73fc9 None admin] PATH_INFO: `/v3/projects/9c9599f863ec41148d52358393690c9c` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:42.586572 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ffadb7-2a35-48a4-a574-e54b0ab73fc9 None admin] RBAC: Authorizing `identity:get_project(project_id=9c9599f863ec41148d52358393690c9c)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:42.588019 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ffadb7-2a35-48a4-a574-e54b0ab73fc9 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:42.589638 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 439/878] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 08:33:42 2022] GET /identity/v3/projects/9c9599f863ec41148d52358393690c9c => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:33:42.595703 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c2050f-e99a-427a-bf9e-89593bd4c672 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:42.672036 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:42.692657 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c2050f-e99a-427a-bf9e-89593bd4c672 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:42.725016 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0d89b4-c822-491f-911d-779ca773d082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:42.725707 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d89b4-c822-491f-911d-779ca773d082 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:42.725990 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d89b4-c822-491f-911d-779ca773d082 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:42.726264 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d89b4-c822-491f-911d-779ca773d082 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:42.726981 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0d89b4-c822-491f-911d-779ca773d082 None admin] RBAC: Adding query filter params (role.id=14615e30ed3f4d8cb78ee5e0bf9b5761, scope.project.id=9c9599f863ec41148d52358393690c9c, user.id=e6c2d2245c0842b58721d49a52c2af92) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:33:42.727287 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0d89b4-c822-491f-911d-779ca773d082 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:42.728682 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0d89b4-c822-491f-911d-779ca773d082 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:42.734204 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 440/879] 10.222.0.30 () {56 vars in 1374 bytes} [Sat Oct 15 08:33:42 2022] GET /identity/v3/role_assignments?user.id=e6c2d2245c0842b58721d49a52c2af92&scope.project.id=9c9599f863ec41148d52358393690c9c&role.id=14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:33:44.856415 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8617002-51a3-455a-ac6a-0618f2a79f04 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:44.857004 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8617002-51a3-455a-ac6a-0618f2a79f04 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:44.857004 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8617002-51a3-455a-ac6a-0618f2a79f04 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:44.858334 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 440/880] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:33:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:33:44.863583 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4317a2d0-4f26-47bd-b843-e0ada915c4f5 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:44.863997 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4317a2d0-4f26-47bd-b843-e0ada915c4f5 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:44.864140 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4317a2d0-4f26-47bd-b843-e0ada915c4f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:44.882337 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-4317a2d0-4f26-47bd-b843-e0ada915c4f5 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:33:44.883513 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-4317a2d0-4f26-47bd-b843-e0ada915c4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:33:44.947313 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 441/881] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:33:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 08:33:45.214822 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8617002-51a3-455a-ac6a-0618f2a79f04 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:45.306874 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:45.314575 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8617002-51a3-455a-ac6a-0618f2a79f04 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:45.347260 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74531948-46bb-47a1-9b1b-d4352663fd41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:45.347855 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74531948-46bb-47a1-9b1b-d4352663fd41 None admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:45.348138 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74531948-46bb-47a1-9b1b-d4352663fd41 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:45.348415 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74531948-46bb-47a1-9b1b-d4352663fd41 None admin] PATH_INFO: `/v3/domains` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:45.349091 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74531948-46bb-47a1-9b1b-d4352663fd41 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:45.351050 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74531948-46bb-47a1-9b1b-d4352663fd41 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:45.367654 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 441/882] 10.222.0.30 () {60 vars in 1138 bytes} [Sat Oct 15 08:33:45 2022] POST /identity/v3/domains => generated 267 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:33:47.508850 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbad921-ed97-4eae-b405-cbf75d720840 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:47.509377 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbad921-ed97-4eae-b405-cbf75d720840 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:47.509483 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbad921-ed97-4eae-b405-cbf75d720840 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:47.510781 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 442/883] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:33:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:33:47.516031 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4a6520-37d3-4499-91f1-a2b3af0f0942 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:47.516481 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4a6520-37d3-4499-91f1-a2b3af0f0942 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:47.516582 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4a6520-37d3-4499-91f1-a2b3af0f0942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:47.535042 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-db4a6520-37d3-4499-91f1-a2b3af0f0942 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:33:47.536217 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-db4a6520-37d3-4499-91f1-a2b3af0f0942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:33:47.600308 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 442/884] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:33:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 08:33:47.870597 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbad921-ed97-4eae-b405-cbf75d720840 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:47.938350 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:47.967885 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbad921-ed97-4eae-b405-cbf75d720840 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:48.000117 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7565e8d0-68bf-43a5-acbe-a97659127d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:48.000727 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7565e8d0-68bf-43a5-acbe-a97659127d1b None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:48.001009 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7565e8d0-68bf-43a5-acbe-a97659127d1b None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:48.001286 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7565e8d0-68bf-43a5-acbe-a97659127d1b None admin] PATH_INFO: `/v3/domains/da5b9c212edb424b8d61c40ea5f1f63e` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:48.011955 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7565e8d0-68bf-43a5-acbe-a97659127d1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=da5b9c212edb424b8d61c40ea5f1f63e)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:48.013378 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7565e8d0-68bf-43a5-acbe-a97659127d1b None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:48.014973 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 443/885] 10.222.0.30 () {56 vars in 1183 bytes} [Sat Oct 15 08:33:47 2022] GET /identity/v3/domains/da5b9c212edb424b8d61c40ea5f1f63e => generated 267 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:33:48.020922 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4a6520-37d3-4499-91f1-a2b3af0f0942 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:48.101117 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:48.117557 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4a6520-37d3-4499-91f1-a2b3af0f0942 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:48.150478 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b72ca75-81ad-471d-865d-76125cb3dc33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:48.151022 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b72ca75-81ad-471d-865d-76125cb3dc33 None admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:48.151300 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b72ca75-81ad-471d-865d-76125cb3dc33 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:48.151574 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b72ca75-81ad-471d-865d-76125cb3dc33 None admin] PATH_INFO: `/v3/projects` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:48.152214 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b72ca75-81ad-471d-865d-76125cb3dc33 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:48.153641 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b72ca75-81ad-471d-865d-76125cb3dc33 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:48.179051 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 443/886] 10.222.0.30 () {60 vars in 1141 bytes} [Sat Oct 15 08:33:48 2022] POST /identity/v3/projects => generated 361 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:33:50.384833 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c07f8a5-d77e-4f7f-b77f-6c03fa63d8a7 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:50.385391 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c07f8a5-d77e-4f7f-b77f-6c03fa63d8a7 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:50.385391 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c07f8a5-d77e-4f7f-b77f-6c03fa63d8a7 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:50.386777 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 444/887] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:33:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:33:50.392026 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5591437b-4b58-4993-8a45-e8c39aafb0f2 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:50.392452 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5591437b-4b58-4993-8a45-e8c39aafb0f2 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:50.392584 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5591437b-4b58-4993-8a45-e8c39aafb0f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:50.411265 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-5591437b-4b58-4993-8a45-e8c39aafb0f2 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:33:50.412460 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-5591437b-4b58-4993-8a45-e8c39aafb0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:33:50.477249 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 444/888] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:33:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 08:33:50.740270 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c07f8a5-d77e-4f7f-b77f-6c03fa63d8a7 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:50.837780 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c07f8a5-d77e-4f7f-b77f-6c03fa63d8a7 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:50.864873 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:50.869979 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626b9940-d03e-48cf-b149-786b988e6965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:50.870598 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626b9940-d03e-48cf-b149-786b988e6965 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:50.870882 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626b9940-d03e-48cf-b149-786b988e6965 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:50.871160 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626b9940-d03e-48cf-b149-786b988e6965 None admin] PATH_INFO: `/v3/domains/da5b9c212edb424b8d61c40ea5f1f63e` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:50.872259 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626b9940-d03e-48cf-b149-786b988e6965 None admin] RBAC: Authorizing `identity:get_domain(domain_id=da5b9c212edb424b8d61c40ea5f1f63e)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:50.873671 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626b9940-d03e-48cf-b149-786b988e6965 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:50.875235 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 445/889] 10.222.0.30 () {56 vars in 1183 bytes} [Sat Oct 15 08:33:50 2022] GET /identity/v3/domains/da5b9c212edb424b8d61c40ea5f1f63e => generated 267 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 08:33:50.880762 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5591437b-4b58-4993-8a45-e8c39aafb0f2 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:50.944246 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:50.978895 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5591437b-4b58-4993-8a45-e8c39aafb0f2 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:51.010726 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01515299-fc57-4e52-a758-32d383d9182f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:51.011323 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01515299-fc57-4e52-a758-32d383d9182f None admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:51.011603 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01515299-fc57-4e52-a758-32d383d9182f None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:51.011874 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01515299-fc57-4e52-a758-32d383d9182f None admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:51.012525 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01515299-fc57-4e52-a758-32d383d9182f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:51.013977 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01515299-fc57-4e52-a758-32d383d9182f None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:51.035156 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 445/890] 10.222.0.30 () {60 vars in 1132 bytes} [Sat Oct 15 08:33:50 2022] POST /identity/v3/users => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:33:53.160636 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f5ecd-fb52-403a-a111-85b28a5d1779 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:53.161201 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f5ecd-fb52-403a-a111-85b28a5d1779 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:53.161201 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f5ecd-fb52-403a-a111-85b28a5d1779 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:53.163042 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 446/891] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:33:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:33:53.168205 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f742229-38e8-4af4-9aac-e67ce4756088 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:53.168628 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f742229-38e8-4af4-9aac-e67ce4756088 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:53.168760 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f742229-38e8-4af4-9aac-e67ce4756088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:53.187772 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-8f742229-38e8-4af4-9aac-e67ce4756088 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:33:53.188957 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-8f742229-38e8-4af4-9aac-e67ce4756088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:33:53.367673 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 446/892] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:33:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 08:33:53.632632 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11f5ecd-fb52-403a-a111-85b28a5d1779 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:53.729026 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11f5ecd-fb52-403a-a111-85b28a5d1779 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:53.761786 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b439f770-8353-47fc-ae75-87a2d413d2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:53.762402 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b439f770-8353-47fc-ae75-87a2d413d2fc None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:53.762681 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b439f770-8353-47fc-ae75-87a2d413d2fc None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:53.762954 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b439f770-8353-47fc-ae75-87a2d413d2fc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:53.773485 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b439f770-8353-47fc-ae75-87a2d413d2fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:33:53.778225 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-b439f770-8353-47fc-ae75-87a2d413d2fc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 08:33:53.778225 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:33:53.778225 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:33:53.778225 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:33:53.778225 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:33:53.778225 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:33:53.778225 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:33:53.778225 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:33:53.778225 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:33:53.778225 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:33:53.778225 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:33:53.778225 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:33:53.778225 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:33:53.778225 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:33:53.778225 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:33:53.778225 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:33:53.779392 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:33:53.779392 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:33:53.779392 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:33:53.779392 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:33:53.779392 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:33:53.779392 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:33:53.779392 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:33:53.779392 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:33:53.779392 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:33:53.779392 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:33:53.779392 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:33:53.779392 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:33:53.779392 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:33:53.779392 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:33:53.779392 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:33:53.779392 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:33:53.780480 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:33:53.780480 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:33:53.780480 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:33:53.780480 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:33:53.780480 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:33:53.780480 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:33:53.780480 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:33:53.780480 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:33:53.780480 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:33:53.780480 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:33:53.780480 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 08:33:53.780480 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:33:53.780480 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 447/893] 10.222.0.30 () {56 vars in 1096 bytes} [Sat Oct 15 08:33:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:33:53.785282 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f742229-38e8-4af4-9aac-e67ce4756088 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:53.835364 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:53.880489 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f742229-38e8-4af4-9aac-e67ce4756088 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:53.913626 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10dee05-883f-42cc-9367-3e06403b7501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:53.914181 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10dee05-883f-42cc-9367-3e06403b7501 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:53.914458 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10dee05-883f-42cc-9367-3e06403b7501 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:53.914727 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10dee05-883f-42cc-9367-3e06403b7501 None admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:53.915376 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10dee05-883f-42cc-9367-3e06403b7501 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:33:53.915675 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10dee05-883f-42cc-9367-3e06403b7501 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:53.917034 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10dee05-883f-42cc-9367-3e06403b7501 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:53.924660 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 447/894] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 08:33:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:33:53.930832 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b439f770-8353-47fc-ae75-87a2d413d2fc None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:53.987739 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:54.026523 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b439f770-8353-47fc-ae75-87a2d413d2fc None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:54.059103 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb298b4-1a3e-4d5b-8027-3fdf3e38527a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:54.059709 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb298b4-1a3e-4d5b-8027-3fdf3e38527a None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:54.059987 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb298b4-1a3e-4d5b-8027-3fdf3e38527a None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:54.060259 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb298b4-1a3e-4d5b-8027-3fdf3e38527a None admin] PATH_INFO: `/v3/users/a20d48625ffa4984944906e73f7e0ad8` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:54.073199 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb298b4-1a3e-4d5b-8027-3fdf3e38527a None admin] RBAC: Authorizing `identity:get_user(user_id=a20d48625ffa4984944906e73f7e0ad8)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:54.074642 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb298b4-1a3e-4d5b-8027-3fdf3e38527a None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:54.076199 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 448/895] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 08:33:53 2022] GET /identity/v3/users/a20d48625ffa4984944906e73f7e0ad8 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:33:54.081851 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10dee05-883f-42cc-9367-3e06403b7501 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:54.176838 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10dee05-883f-42cc-9367-3e06403b7501 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:54.190289 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:54.210476 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf321a4-ed47-48c2-b15f-7f3bce439948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:54.211087 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf321a4-ed47-48c2-b15f-7f3bce439948 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:54.211367 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf321a4-ed47-48c2-b15f-7f3bce439948 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:54.211639 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf321a4-ed47-48c2-b15f-7f3bce439948 None admin] PATH_INFO: `/v3/projects/611794d909184a1eb6c1dd151487b1a4` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:54.212727 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf321a4-ed47-48c2-b15f-7f3bce439948 None admin] RBAC: Authorizing `identity:get_project(project_id=611794d909184a1eb6c1dd151487b1a4)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:54.214175 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf321a4-ed47-48c2-b15f-7f3bce439948 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:54.215765 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 448/896] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 08:33:54 2022] GET /identity/v3/projects/611794d909184a1eb6c1dd151487b1a4 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:33:54.221408 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb298b4-1a3e-4d5b-8027-3fdf3e38527a None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:54.315734 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb298b4-1a3e-4d5b-8027-3fdf3e38527a None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:54.333325 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:54.349709 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddce48e-651a-42e8-bb42-836d9d45f7a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:54.350863 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddce48e-651a-42e8-bb42-836d9d45f7a7 None admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:54.351145 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddce48e-651a-42e8-bb42-836d9d45f7a7 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:54.351421 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddce48e-651a-42e8-bb42-836d9d45f7a7 None admin] PATH_INFO: `/v3/projects/611794d909184a1eb6c1dd151487b1a4/users/a20d48625ffa4984944906e73f7e0ad8/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:54.353289 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddce48e-651a-42e8-bb42-836d9d45f7a7 None admin] RBAC: Authorizing `identity:create_grant(project_id=611794d909184a1eb6c1dd151487b1a4, user_id=a20d48625ffa4984944906e73f7e0ad8, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:54.354740 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddce48e-651a-42e8-bb42-836d9d45f7a7 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:54.367148 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 449/897] 10.222.0.30 () {58 vars in 1439 bytes} [Sat Oct 15 08:33:54 2022] PUT /identity/v3/projects/611794d909184a1eb6c1dd151487b1a4/users/a20d48625ffa4984944906e73f7e0ad8/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 08:33:56.461431 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cee6459-63fe-453c-8261-57bf26316468 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:56.461973 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cee6459-63fe-453c-8261-57bf26316468 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:56.462066 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cee6459-63fe-453c-8261-57bf26316468 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:56.463334 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 449/898] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:33:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 08:33:56.468464 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a056775-715a-4c23-8474-52730b4d07f2 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:56.468869 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a056775-715a-4c23-8474-52730b4d07f2 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:56.469023 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a056775-715a-4c23-8474-52730b4d07f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:56.487167 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-8a056775-715a-4c23-8474-52730b4d07f2 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:33:56.488351 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-8a056775-715a-4c23-8474-52730b4d07f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:33:56.551994 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 450/899] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:33:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 08:33:56.816340 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cee6459-63fe-453c-8261-57bf26316468 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:56.915755 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cee6459-63fe-453c-8261-57bf26316468 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:56.948668 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d375fbb-80ab-4abf-9bc3-719a9252d43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:56.949307 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d375fbb-80ab-4abf-9bc3-719a9252d43a None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:56.949598 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d375fbb-80ab-4abf-9bc3-719a9252d43a None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:56.949893 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d375fbb-80ab-4abf-9bc3-719a9252d43a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:56.960271 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d375fbb-80ab-4abf-9bc3-719a9252d43a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:33:56.965020 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-0d375fbb-80ab-4abf-9bc3-719a9252d43a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 08:33:56.965020 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:33:56.965020 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:33:56.965020 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:33:56.965020 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:33:56.965020 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:33:56.965020 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:33:56.965020 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:33:56.965020 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:33:56.965020 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:33:56.965020 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:33:56.965020 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:33:56.965020 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:33:56.965020 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:33:56.965020 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:33:56.965020 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:33:56.966120 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:33:56.966120 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:33:56.966120 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:33:56.966120 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:33:56.966120 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:33:56.966120 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:33:56.966120 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:33:56.966120 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:33:56.966120 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:33:56.966120 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:33:56.966120 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:33:56.966120 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:33:56.966120 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:33:56.966120 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:33:56.966120 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:33:56.966120 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:33:56.967325 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:33:56.967325 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:33:56.967325 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:33:56.967325 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:33:56.967325 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:33:56.967325 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:33:56.967325 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:33:56.967325 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:33:56.967325 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:33:56.967325 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:33:56.967325 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 08:33:56.967325 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:33:56.967325 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 450/900] 10.222.0.30 () {56 vars in 1096 bytes} [Sat Oct 15 08:33:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:33:56.971931 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a056775-715a-4c23-8474-52730b4d07f2 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:57.067615 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a056775-715a-4c23-8474-52730b4d07f2 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:57.076994 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:57.101504 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dde6f1a-94e3-4ca1-a35c-af451ad75ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:57.102126 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde6f1a-94e3-4ca1-a35c-af451ad75ad3 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:57.102413 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde6f1a-94e3-4ca1-a35c-af451ad75ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:57.102689 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde6f1a-94e3-4ca1-a35c-af451ad75ad3 None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:57.103347 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dde6f1a-94e3-4ca1-a35c-af451ad75ad3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:33:57.103654 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dde6f1a-94e3-4ca1-a35c-af451ad75ad3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:57.105002 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dde6f1a-94e3-4ca1-a35c-af451ad75ad3 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:57.112810 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 451/901] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 08:33:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:33:57.119852 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d375fbb-80ab-4abf-9bc3-719a9252d43a None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:57.161253 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:57.215405 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d375fbb-80ab-4abf-9bc3-719a9252d43a None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:57.254121 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3886a518-7339-4213-98c5-0421b0adda45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:57.254739 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3886a518-7339-4213-98c5-0421b0adda45 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:57.255021 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3886a518-7339-4213-98c5-0421b0adda45 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:57.255297 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3886a518-7339-4213-98c5-0421b0adda45 None admin] PATH_INFO: `/v3/users/a20d48625ffa4984944906e73f7e0ad8` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:57.257051 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3886a518-7339-4213-98c5-0421b0adda45 None admin] RBAC: Authorizing `identity:get_user(user_id=a20d48625ffa4984944906e73f7e0ad8)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:57.258506 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3886a518-7339-4213-98c5-0421b0adda45 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:57.260083 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 451/902] 10.222.0.30 () {56 vars in 1177 bytes} [Sat Oct 15 08:33:57 2022] GET /identity/v3/users/a20d48625ffa4984944906e73f7e0ad8 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:33:57.265832 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dde6f1a-94e3-4ca1-a35c-af451ad75ad3 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:57.361521 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dde6f1a-94e3-4ca1-a35c-af451ad75ad3 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:57.393976 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c34891-e5c6-4d5b-ad6b-e3493cc9a9a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:57.394581 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c34891-e5c6-4d5b-ad6b-e3493cc9a9a4 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:57.394867 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c34891-e5c6-4d5b-ad6b-e3493cc9a9a4 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:57.395147 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c34891-e5c6-4d5b-ad6b-e3493cc9a9a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:57.432731 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:57.461413 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c34891-e5c6-4d5b-ad6b-e3493cc9a9a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:57.462886 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c34891-e5c6-4d5b-ad6b-e3493cc9a9a4 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:57.555687 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 452/903] 10.222.0.30 () {58 vars in 1282 bytes} [Sat Oct 15 08:33:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 15 08:33:57.561574 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3886a518-7339-4213-98c5-0421b0adda45 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:57.657286 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3886a518-7339-4213-98c5-0421b0adda45 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:57.668004 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:57.690161 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2feb87-4f99-49f4-854b-4d11bec717db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:57.690774 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2feb87-4f99-49f4-854b-4d11bec717db None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:57.691053 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2feb87-4f99-49f4-854b-4d11bec717db None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:57.691325 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2feb87-4f99-49f4-854b-4d11bec717db None admin] PATH_INFO: `/v3/projects/611794d909184a1eb6c1dd151487b1a4` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:57.692416 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2feb87-4f99-49f4-854b-4d11bec717db None admin] RBAC: Authorizing `identity:get_project(project_id=611794d909184a1eb6c1dd151487b1a4)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:57.694416 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2feb87-4f99-49f4-854b-4d11bec717db None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:57.696011 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 452/904] 10.222.0.30 () {56 vars in 1186 bytes} [Sat Oct 15 08:33:57 2022] GET /identity/v3/projects/611794d909184a1eb6c1dd151487b1a4 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:33:57.702181 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c34891-e5c6-4d5b-ad6b-e3493cc9a9a4 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:33:57.768119 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:33:57.798036 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c34891-e5c6-4d5b-ad6b-e3493cc9a9a4 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:33:57.830753 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc9a5ef-3575-4ecc-88c2-dcaaef17d49e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:33:57.831339 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc9a5ef-3575-4ecc-88c2-dcaaef17d49e None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:33:57.831617 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc9a5ef-3575-4ecc-88c2-dcaaef17d49e None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:33:57.831889 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc9a5ef-3575-4ecc-88c2-dcaaef17d49e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:33:57.832598 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc9a5ef-3575-4ecc-88c2-dcaaef17d49e None admin] RBAC: Adding query filter params (role.id=14615e30ed3f4d8cb78ee5e0bf9b5761, scope.project.id=611794d909184a1eb6c1dd151487b1a4, user.id=a20d48625ffa4984944906e73f7e0ad8) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:33:57.832904 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc9a5ef-3575-4ecc-88c2-dcaaef17d49e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:33:57.834376 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc9a5ef-3575-4ecc-88c2-dcaaef17d49e None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:33:57.839939 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 453/905] 10.222.0.30 () {56 vars in 1374 bytes} [Sat Oct 15 08:33:57 2022] GET /identity/v3/role_assignments?user.id=a20d48625ffa4984944906e73f7e0ad8&scope.project.id=611794d909184a1eb6c1dd151487b1a4&role.id=14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:34:07.686277 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=11 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:34:07.837372 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=33 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:34:53.527238 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1893746d-e6b7-4f8d-8c0c-23df0d7bcda0 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:34:53.527854 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1893746d-e6b7-4f8d-8c0c-23df0d7bcda0 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:34:53.527854 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1893746d-e6b7-4f8d-8c0c-23df0d7bcda0 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:34:53.529318 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 453/906] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:34:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 08:34:53.535870 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd011af-2e3c-413e-af1f-97ba50eaa25c None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:34:53.536403 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd011af-2e3c-413e-af1f-97ba50eaa25c None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:34:53.536612 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd011af-2e3c-413e-af1f-97ba50eaa25c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:34:53.559861 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-ecd011af-2e3c-413e-af1f-97ba50eaa25c None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:34:53.561073 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-ecd011af-2e3c-413e-af1f-97ba50eaa25c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:34:53.650592 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 454/907] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:34:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 08:34:53.946266 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1893746d-e6b7-4f8d-8c0c-23df0d7bcda0 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:34:54.079549 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1893746d-e6b7-4f8d-8c0c-23df0d7bcda0 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:34:54.137291 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70078ea4-b426-454e-bc23-e64058d798ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:34:54.138006 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70078ea4-b426-454e-bc23-e64058d798ee None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:34:54.138279 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70078ea4-b426-454e-bc23-e64058d798ee None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:34:54.138599 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70078ea4-b426-454e-bc23-e64058d798ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:34:54.150711 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70078ea4-b426-454e-bc23-e64058d798ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:34:54.152309 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70078ea4-b426-454e-bc23-e64058d798ee None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:34:54.169823 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-70078ea4-b426-454e-bc23-e64058d798ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:34:54.169823 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:34:54.169823 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:34:54.169823 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:34:54.169823 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:34:54.169823 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:34:54.169823 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:34:54.169823 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:34:54.169823 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:34:54.169823 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:34:54.169823 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:34:54.169823 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:34:54.169823 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 08:34:54.169823 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 08:34:54.169823 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 08:34:54.169823 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 08:34:54.171339 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:34:54.171339 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:34:54.171339 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:34:54.171339 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:34:54.171339 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:34:54.171339 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:34:54.171339 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:34:54.171339 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:34:54.171339 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:34:54.171339 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:34:54.171339 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:34:54.171339 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:34:54.171339 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:34:54.171339 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:34:54.171339 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:34:54.171339 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:34:54.171339 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 08:34:54.172560 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 08:34:54.172560 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:34:54.172560 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:34:54.176597 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 454/908] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:34:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:34:54.178504 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd011af-2e3c-413e-af1f-97ba50eaa25c None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:34:54.304368 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd011af-2e3c-413e-af1f-97ba50eaa25c None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:34:54.339637 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:34:54.347689 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03af6f30-f42a-4bbd-ab38-42accbe6989d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:34:54.348332 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af6f30-f42a-4bbd-ab38-42accbe6989d None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:34:54.348633 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af6f30-f42a-4bbd-ab38-42accbe6989d None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:34:54.349493 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af6f30-f42a-4bbd-ab38-42accbe6989d None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:34:54.350276 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03af6f30-f42a-4bbd-ab38-42accbe6989d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:34:54.350776 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03af6f30-f42a-4bbd-ab38-42accbe6989d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:34:54.352625 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03af6f30-f42a-4bbd-ab38-42accbe6989d None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:34:54.368603 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 455/909] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 08:34:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:34:54.374712 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70078ea4-b426-454e-bc23-e64058d798ee None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:34:54.459988 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:34:54.530640 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70078ea4-b426-454e-bc23-e64058d798ee None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:34:54.582555 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d58e00-494b-4a8e-b4ad-8d4a5925c11d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:34:54.583411 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d58e00-494b-4a8e-b4ad-8d4a5925c11d None admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:34:54.583911 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d58e00-494b-4a8e-b4ad-8d4a5925c11d None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:34:54.584359 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d58e00-494b-4a8e-b4ad-8d4a5925c11d None admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:34:54.585268 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d58e00-494b-4a8e-b4ad-8d4a5925c11d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:34:54.587168 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d58e00-494b-4a8e-b4ad-8d4a5925c11d None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:34:54.620213 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 455/910] 10.222.0.30 () {60 vars in 1132 bytes} [Sat Oct 15 08:34:54 2022] POST /identity/v3/users => generated 251 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:34:56.874609 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e37bd-0d78-44ae-9078-779ad85e2442 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:34:56.875234 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e37bd-0d78-44ae-9078-779ad85e2442 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:34:56.875234 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e37bd-0d78-44ae-9078-779ad85e2442 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:34:56.876560 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 456/911] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:34:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:34:56.882517 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e8955-b8dc-4238-b1c3-f1c445df41ea None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:34:56.883089 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e8955-b8dc-4238-b1c3-f1c445df41ea None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:34:56.883089 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e8955-b8dc-4238-b1c3-f1c445df41ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:34:56.902024 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-d38e8955-b8dc-4238-b1c3-f1c445df41ea None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:34:56.903254 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-d38e8955-b8dc-4238-b1c3-f1c445df41ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:34:56.978167 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 456/912] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:34:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 08:34:57.259982 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e37bd-0d78-44ae-9078-779ad85e2442 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:34:57.380544 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e37bd-0d78-44ae-9078-779ad85e2442 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:34:57.435389 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9c8bbd-c921-4908-9f0f-98fb2a5f1e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:34:57.436083 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9c8bbd-c921-4908-9f0f-98fb2a5f1e83 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:34:57.436368 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9c8bbd-c921-4908-9f0f-98fb2a5f1e83 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:34:57.436643 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9c8bbd-c921-4908-9f0f-98fb2a5f1e83 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:34:57.447698 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9c8bbd-c921-4908-9f0f-98fb2a5f1e83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:34:57.452638 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-bb9c8bbd-c921-4908-9f0f-98fb2a5f1e83 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 15 08:34:57.452638 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:34:57.452638 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:34:57.452638 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:34:57.452638 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:34:57.452638 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:34:57.452638 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:34:57.452638 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:34:57.452638 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:34:57.452638 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:34:57.452638 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:34:57.452638 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:34:57.452638 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:34:57.452638 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:34:57.452638 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:34:57.452638 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:34:57.453625 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:34:57.453625 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:34:57.453625 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:34:57.453625 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:34:57.453625 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:34:57.453625 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:34:57.453625 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:34:57.453625 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:34:57.453625 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:34:57.453625 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:34:57.453625 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:34:57.453625 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:34:57.453625 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:34:57.453625 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:34:57.453625 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:34:57.453625 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:34:57.454583 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:34:57.454583 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:34:57.454583 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:34:57.454583 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:34:57.454583 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:34:57.454583 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:34:57.454583 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:34:57.454583 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:34:57.454583 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:34:57.454583 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:34:57.454583 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 15 08:34:57.454583 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:34:57.458850 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 457/913] 10.222.0.30 () {56 vars in 1102 bytes} [Sat Oct 15 08:34:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:34:57.460170 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38e8955-b8dc-4238-b1c3-f1c445df41ea None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:34:57.507507 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:34:57.573566 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38e8955-b8dc-4238-b1c3-f1c445df41ea None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:34:57.612129 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afec89e6-969c-4a03-a493-4d8b4fef7f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:34:57.613202 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec89e6-969c-4a03-a493-4d8b4fef7f18 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:34:57.613489 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec89e6-969c-4a03-a493-4d8b4fef7f18 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:34:57.613783 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec89e6-969c-4a03-a493-4d8b4fef7f18 None admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:34:57.614596 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afec89e6-969c-4a03-a493-4d8b4fef7f18 None admin] RBAC: Adding query filter params (name=service) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:34:57.614909 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afec89e6-969c-4a03-a493-4d8b4fef7f18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:34:57.616387 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afec89e6-969c-4a03-a493-4d8b4fef7f18 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:34:57.625183 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 457/914] 10.222.0.30 () {56 vars in 1103 bytes} [Sat Oct 15 08:34:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:34:57.632577 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9c8bbd-c921-4908-9f0f-98fb2a5f1e83 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:34:57.695191 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:34:57.750175 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9c8bbd-c921-4908-9f0f-98fb2a5f1e83 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:34:57.787513 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10340ae2-3a5e-4850-8f3b-b05e8a85d8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:34:57.788147 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10340ae2-3a5e-4850-8f3b-b05e8a85d8f8 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:34:57.788434 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10340ae2-3a5e-4850-8f3b-b05e8a85d8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:34:57.788742 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10340ae2-3a5e-4850-8f3b-b05e8a85d8f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:34:57.799436 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10340ae2-3a5e-4850-8f3b-b05e8a85d8f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:34:57.800873 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10340ae2-3a5e-4850-8f3b-b05e8a85d8f8 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:34:57.815621 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-10340ae2-3a5e-4850-8f3b-b05e8a85d8f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:34:57.815621 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:34:57.815621 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:34:57.815621 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:34:57.815621 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:34:57.815621 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:34:57.815621 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:34:57.815621 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:34:57.815621 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:34:57.815621 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:34:57.815621 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:34:57.815621 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:34:57.815621 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 08:34:57.815621 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 08:34:57.815621 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 08:34:57.815621 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 08:34:57.821546 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:34:57.821546 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:34:57.821546 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:34:57.821546 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:34:57.821546 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:34:57.821546 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:34:57.821546 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:34:57.821546 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:34:57.821546 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:34:57.821546 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:34:57.821546 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:34:57.821546 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:34:57.821546 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:34:57.821546 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:34:57.821546 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:34:57.821546 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:34:57.821546 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 08:34:57.822491 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 08:34:57.822491 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:34:57.822491 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:34:57.822491 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 458/915] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:34:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:34:57.822728 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afec89e6-969c-4a03-a493-4d8b4fef7f18 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:34:57.932958 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:34:57.937551 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afec89e6-969c-4a03-a493-4d8b4fef7f18 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:34:57.974565 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1817295c-037a-4004-9f17-3d0fb8247a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:34:57.975176 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1817295c-037a-4004-9f17-3d0fb8247a83 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:34:57.975455 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1817295c-037a-4004-9f17-3d0fb8247a83 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:34:57.975730 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1817295c-037a-4004-9f17-3d0fb8247a83 None admin] PATH_INFO: `/v3/domains` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:34:57.976376 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1817295c-037a-4004-9f17-3d0fb8247a83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:34:57.976733 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1817295c-037a-4004-9f17-3d0fb8247a83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:34:57.978236 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1817295c-037a-4004-9f17-3d0fb8247a83 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:34:57.991942 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 458/916] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 08:34:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:34:57.998303 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10340ae2-3a5e-4850-8f3b-b05e8a85d8f8 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:34:58.112815 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10340ae2-3a5e-4850-8f3b-b05e8a85d8f8 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:34:58.114845 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:34:58.160355 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d627870c-647d-4baf-a451-406886653e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:34:58.161096 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d627870c-647d-4baf-a451-406886653e69 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:34:58.162034 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d627870c-647d-4baf-a451-406886653e69 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:34:58.162367 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d627870c-647d-4baf-a451-406886653e69 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:34:58.170583 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d627870c-647d-4baf-a451-406886653e69 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:34:58.172104 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d627870c-647d-4baf-a451-406886653e69 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:34:58.186159 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-d627870c-647d-4baf-a451-406886653e69 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 15 08:34:58.186159 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:34:58.186159 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:34:58.186159 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:34:58.186159 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:34:58.186159 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:34:58.186159 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:34:58.186159 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:34:58.186159 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:34:58.186159 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:34:58.186159 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:34:58.186159 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:34:58.186159 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 08:34:58.186159 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 08:34:58.186159 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 08:34:58.186159 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 08:34:58.187170 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:34:58.187170 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:34:58.187170 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 08:34:58.187170 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 08:34:58.187170 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 08:34:58.187170 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 08:34:58.187170 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:34:58.187170 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:34:58.187170 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:34:58.187170 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:34:58.187170 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:34:58.187170 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:34:58.187170 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:34:58.187170 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:34:58.187170 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:34:58.187170 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:34:58.187170 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:34:58.187170 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:34:58.188158 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:34:58.188158 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:34:58.188158 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 08:34:58.188158 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 08:34:58.188158 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 08:34:58.188158 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 08:34:58.188158 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 08:34:58.188158 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 08:34:58.188158 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 15 08:34:58.188158 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:34:58.189836 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 459/917] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:34:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:34:58.195153 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1817295c-037a-4004-9f17-3d0fb8247a83 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:34:58.315059 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1817295c-037a-4004-9f17-3d0fb8247a83 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:34:58.358604 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07da0345-d7a1-4234-9456-6ae6ee54ff9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:34:58.359333 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da0345-d7a1-4234-9456-6ae6ee54ff9f None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:34:58.359657 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da0345-d7a1-4234-9456-6ae6ee54ff9f None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:34:58.359986 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da0345-d7a1-4234-9456-6ae6ee54ff9f None admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:34:58.360883 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07da0345-d7a1-4234-9456-6ae6ee54ff9f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:34:58.361294 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07da0345-d7a1-4234-9456-6ae6ee54ff9f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:34:58.363113 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07da0345-d7a1-4234-9456-6ae6ee54ff9f None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:34:58.392783 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 459/918] 10.222.0.30 () {56 vars in 1143 bytes} [Sat Oct 15 08:34:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:34:58.394352 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d627870c-647d-4baf-a451-406886653e69 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:34:58.518215 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d627870c-647d-4baf-a451-406886653e69 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:34:58.556929 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38ca49e-8aaf-470c-9900-e91f096d6091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:34:58.557561 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ca49e-8aaf-470c-9900-e91f096d6091 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:34:58.557887 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ca49e-8aaf-470c-9900-e91f096d6091 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:34:58.558161 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ca49e-8aaf-470c-9900-e91f096d6091 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:34:58.569077 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38ca49e-8aaf-470c-9900-e91f096d6091 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:34:58.570555 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38ca49e-8aaf-470c-9900-e91f096d6091 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:34:58.586693 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-d38ca49e-8aaf-470c-9900-e91f096d6091 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:34:58.586693 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:34:58.586693 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:34:58.586693 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:34:58.586693 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:34:58.586693 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:34:58.586693 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:34:58.586693 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:34:58.586693 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:34:58.586693 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:34:58.586693 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:34:58.586693 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:34:58.586693 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 08:34:58.586693 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 08:34:58.586693 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 08:34:58.586693 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 08:34:58.587923 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:34:58.587923 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:34:58.587923 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:34:58.587923 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:34:58.587923 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:34:58.587923 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:34:58.587923 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:34:58.587923 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:34:58.587923 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:34:58.587923 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:34:58.587923 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:34:58.587923 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:34:58.587923 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:34:58.587923 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:34:58.587923 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:34:58.587923 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:34:58.587923 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 08:34:58.589016 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 08:34:58.589016 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:34:58.589016 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:34:58.589016 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 460/919] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:34:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:34:58.593713 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07da0345-d7a1-4234-9456-6ae6ee54ff9f None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:34:58.618169 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:34:58.711377 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07da0345-d7a1-4234-9456-6ae6ee54ff9f None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:34:58.752426 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ab99ae-f8a1-4f97-b2b7-e8994380e870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:34:58.753036 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab99ae-f8a1-4f97-b2b7-e8994380e870 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:34:58.753330 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab99ae-f8a1-4f97-b2b7-e8994380e870 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:34:58.753599 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab99ae-f8a1-4f97-b2b7-e8994380e870 None admin] PATH_INFO: `/v3/domains` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:34:58.754288 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ab99ae-f8a1-4f97-b2b7-e8994380e870 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:34:58.754622 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ab99ae-f8a1-4f97-b2b7-e8994380e870 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:34:58.756510 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ab99ae-f8a1-4f97-b2b7-e8994380e870 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:34:58.768899 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 460/920] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 08:34:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:34:58.777010 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38ca49e-8aaf-470c-9900-e91f096d6091 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:34:58.798320 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:34:58.900330 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38ca49e-8aaf-470c-9900-e91f096d6091 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:34:58.936555 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b9bfe5-eac4-4d9c-8632-49235ff11f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:34:58.937211 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b9bfe5-eac4-4d9c-8632-49235ff11f67 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:34:58.937497 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b9bfe5-eac4-4d9c-8632-49235ff11f67 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:34:58.937808 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b9bfe5-eac4-4d9c-8632-49235ff11f67 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:34:58.945810 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b9bfe5-eac4-4d9c-8632-49235ff11f67 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:34:58.947247 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b9bfe5-eac4-4d9c-8632-49235ff11f67 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:34:58.958744 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-63b9bfe5-eac4-4d9c-8632-49235ff11f67 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 08:34:58.958744 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:34:58.958744 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:34:58.958744 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:34:58.958744 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:34:58.958744 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:34:58.958744 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:34:58.958744 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:34:58.958744 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:34:58.958744 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:34:58.958744 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:34:58.958744 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:34:58.958744 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 08:34:58.958744 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 08:34:58.958744 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 08:34:58.959818 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 08:34:58.959818 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:34:58.959818 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:34:58.959818 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:34:58.959818 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:34:58.959818 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:34:58.959818 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:34:58.959818 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:34:58.959818 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:34:58.959818 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:34:58.959818 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:34:58.959818 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:34:58.959818 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:34:58.959818 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:34:58.959818 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:34:58.959818 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:34:58.959818 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:34:58.965768 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 08:34:58.965768 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 08:34:58.965768 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 08:34:58.965768 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 08:34:58.965768 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 08:34:58.965768 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 08:34:58.965768 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 08:34:58.965768 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:34:58.965768 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 461/921] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 08:34:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:34:58.966296 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ab99ae-f8a1-4f97-b2b7-e8994380e870 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:34:59.044650 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:34:59.066667 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ab99ae-f8a1-4f97-b2b7-e8994380e870 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:34:59.107041 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970cc1cf-5875-4b0b-acef-81b2e6158b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:34:59.107715 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970cc1cf-5875-4b0b-acef-81b2e6158b18 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:34:59.108006 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970cc1cf-5875-4b0b-acef-81b2e6158b18 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:34:59.108313 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970cc1cf-5875-4b0b-acef-81b2e6158b18 None admin] PATH_INFO: `/v3/projects` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:34:59.109009 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970cc1cf-5875-4b0b-acef-81b2e6158b18 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:34:59.109315 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970cc1cf-5875-4b0b-acef-81b2e6158b18 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:34:59.110836 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970cc1cf-5875-4b0b-acef-81b2e6158b18 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:34:59.123373 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 461/922] 10.222.0.30 () {56 vars in 1148 bytes} [Sat Oct 15 08:34:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:34:59.130519 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b9bfe5-eac4-4d9c-8632-49235ff11f67 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:34:59.212158 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:34:59.237668 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b9bfe5-eac4-4d9c-8632-49235ff11f67 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:34:59.277381 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ab5c5a-0133-4532-963d-d21d0f3ef7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:34:59.278077 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ab5c5a-0133-4532-963d-d21d0f3ef7f5 None admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:34:59.278362 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ab5c5a-0133-4532-963d-d21d0f3ef7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:34:59.278676 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ab5c5a-0133-4532-963d-d21d0f3ef7f5 None admin] PATH_INFO: `/v3/projects/5dfceb59b3794439a4586b3cbe408f0e/users/38986ca2612c4fc59d960cd693834f9b/roles/b1b0b29beb794324826a4c4003ae293b` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:34:59.293068 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ab5c5a-0133-4532-963d-d21d0f3ef7f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=5dfceb59b3794439a4586b3cbe408f0e, user_id=38986ca2612c4fc59d960cd693834f9b, role_id=b1b0b29beb794324826a4c4003ae293b)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:34:59.294534 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ab5c5a-0133-4532-963d-d21d0f3ef7f5 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:34:59.307169 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 462/923] 10.222.0.30 () {58 vars in 1439 bytes} [Sat Oct 15 08:34:59 2022] PUT /identity/v3/projects/5dfceb59b3794439a4586b3cbe408f0e/users/38986ca2612c4fc59d960cd693834f9b/roles/b1b0b29beb794324826a4c4003ae293b => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:35:01.471042 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b360b6b-0263-4913-8fbd-35d64951528a None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:35:01.471652 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b360b6b-0263-4913-8fbd-35d64951528a None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:35:01.471652 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b360b6b-0263-4913-8fbd-35d64951528a None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:35:01.473125 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 462/924] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:35:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 08:35:01.479431 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30518cf4-a941-4025-b81d-4bf1e3592157 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:35:01.479873 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30518cf4-a941-4025-b81d-4bf1e3592157 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:35:01.479980 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30518cf4-a941-4025-b81d-4bf1e3592157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:35:01.499144 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-30518cf4-a941-4025-b81d-4bf1e3592157 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:35:01.500339 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-30518cf4-a941-4025-b81d-4bf1e3592157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:35:01.573060 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 463/925] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:35:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 08:35:01.883716 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b360b6b-0263-4913-8fbd-35d64951528a None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:35:02.006051 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b360b6b-0263-4913-8fbd-35d64951528a None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:35:02.044396 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded2dfe1-bfd0-4d37-8e05-218748db347e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:35:02.045030 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded2dfe1-bfd0-4d37-8e05-218748db347e None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:35:02.045317 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded2dfe1-bfd0-4d37-8e05-218748db347e None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:35:02.045598 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded2dfe1-bfd0-4d37-8e05-218748db347e None admin] PATH_INFO: `/v3/roles/service` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:35:02.054472 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:35:02.058663 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded2dfe1-bfd0-4d37-8e05-218748db347e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:35:02.063869 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-ded2dfe1-bfd0-4d37-8e05-218748db347e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 15 08:35:02.063869 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:35:02.063869 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:35:02.063869 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:35:02.063869 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:35:02.063869 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:35:02.063869 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:35:02.063869 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:35:02.063869 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:35:02.063869 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:35:02.063869 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:35:02.063869 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:35:02.063869 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:35:02.063869 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:35:02.063869 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:35:02.063869 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:35:02.065051 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:35:02.065051 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:35:02.065051 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:35:02.065051 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:35:02.065051 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:35:02.065051 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:35:02.065051 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:35:02.065051 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:35:02.065051 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:35:02.065051 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:35:02.065051 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:35:02.065051 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:35:02.065051 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:35:02.065051 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:35:02.065051 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:35:02.065051 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:35:02.065968 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:35:02.065968 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:35:02.065968 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:35:02.065968 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:35:02.065968 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:35:02.065968 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:35:02.065968 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:35:02.065968 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:35:02.065968 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:35:02.065968 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:35:02.065968 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 15 08:35:02.065968 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:35:02.065968 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 463/926] 10.222.0.30 () {56 vars in 1102 bytes} [Sat Oct 15 08:35:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:35:02.073253 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30518cf4-a941-4025-b81d-4bf1e3592157 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:35:02.132465 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:35:02.187307 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30518cf4-a941-4025-b81d-4bf1e3592157 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:35:02.227731 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d95369-2456-4059-9739-677b664fd8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:35:02.228344 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d95369-2456-4059-9739-677b664fd8ce None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:35:02.228628 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d95369-2456-4059-9739-677b664fd8ce None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:35:02.228904 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d95369-2456-4059-9739-677b664fd8ce None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:35:02.229577 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d95369-2456-4059-9739-677b664fd8ce None admin] RBAC: Adding query filter params (name=service) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:35:02.229907 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d95369-2456-4059-9739-677b664fd8ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:35:02.231278 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d95369-2456-4059-9739-677b664fd8ce None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:35:02.240259 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 464/927] 10.222.0.30 () {56 vars in 1103 bytes} [Sat Oct 15 08:35:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:35:02.245962 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded2dfe1-bfd0-4d37-8e05-218748db347e None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:35:02.354186 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded2dfe1-bfd0-4d37-8e05-218748db347e None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:35:02.392638 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bf4dbb-2966-4e60-b96e-1606a7e433ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:35:02.393334 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bf4dbb-2966-4e60-b96e-1606a7e433ea None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:35:02.393607 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bf4dbb-2966-4e60-b96e-1606a7e433ea None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:35:02.394048 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bf4dbb-2966-4e60-b96e-1606a7e433ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:35:02.406688 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bf4dbb-2966-4e60-b96e-1606a7e433ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:35:02.408159 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bf4dbb-2966-4e60-b96e-1606a7e433ea None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:35:02.426144 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-c3bf4dbb-2966-4e60-b96e-1606a7e433ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:35:02.426144 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:35:02.426144 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:35:02.426144 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:35:02.426144 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:35:02.426144 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:35:02.426144 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:35:02.426144 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:35:02.426144 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:35:02.426144 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:35:02.426144 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:35:02.426144 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:35:02.426144 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 08:35:02.426144 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 08:35:02.426144 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 08:35:02.426144 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 08:35:02.427358 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:35:02.427358 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:35:02.427358 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:35:02.427358 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:35:02.427358 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:35:02.427358 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:35:02.427358 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:35:02.427358 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:35:02.427358 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:35:02.427358 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:35:02.427358 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:35:02.427358 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:35:02.427358 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:35:02.427358 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:35:02.427358 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:35:02.427358 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:35:02.427358 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 08:35:02.428610 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 08:35:02.428610 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:35:02.428610 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:35:02.428610 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 464/928] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:35:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:35:02.433619 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d95369-2456-4059-9739-677b664fd8ce None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:35:02.543944 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d95369-2456-4059-9739-677b664fd8ce None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:35:02.562292 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:35:02.582091 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffe6c33-57cc-4d38-b519-696679ff2dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:35:02.582813 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe6c33-57cc-4d38-b519-696679ff2dca None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:35:02.583149 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe6c33-57cc-4d38-b519-696679ff2dca None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:35:02.583408 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe6c33-57cc-4d38-b519-696679ff2dca None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:35:02.584118 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffe6c33-57cc-4d38-b519-696679ff2dca None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:35:02.584427 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffe6c33-57cc-4d38-b519-696679ff2dca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:35:02.585994 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffe6c33-57cc-4d38-b519-696679ff2dca None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:35:02.600178 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 465/929] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 08:35:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:35:02.607019 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bf4dbb-2966-4e60-b96e-1606a7e433ea None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:35:02.672518 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:35:02.737227 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bf4dbb-2966-4e60-b96e-1606a7e433ea None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:35:02.773385 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2ea83d-e0c5-4570-865a-d8294a05d04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:35:02.774044 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2ea83d-e0c5-4570-865a-d8294a05d04d None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:35:02.774332 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2ea83d-e0c5-4570-865a-d8294a05d04d None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:35:02.774639 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2ea83d-e0c5-4570-865a-d8294a05d04d None admin] PATH_INFO: `/v3/users/placement` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:35:02.782279 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2ea83d-e0c5-4570-865a-d8294a05d04d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:35:02.783725 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2ea83d-e0c5-4570-865a-d8294a05d04d None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:35:02.794778 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-9c2ea83d-e0c5-4570-865a-d8294a05d04d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 15 08:35:02.794778 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:35:02.794778 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:35:02.794778 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:35:02.794778 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:35:02.794778 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:35:02.794778 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:35:02.794778 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:35:02.794778 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:35:02.794778 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:35:02.794778 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:35:02.794778 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:35:02.794778 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 08:35:02.794778 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 08:35:02.794778 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 08:35:02.794778 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 08:35:02.795937 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:35:02.795937 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:35:02.795937 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 08:35:02.795937 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 08:35:02.795937 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 08:35:02.795937 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 08:35:02.795937 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:35:02.795937 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:35:02.795937 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:35:02.795937 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:35:02.795937 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:35:02.795937 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:35:02.795937 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:35:02.795937 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:35:02.795937 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:35:02.795937 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:35:02.795937 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:35:02.795937 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:35:02.796952 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:35:02.796952 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:35:02.796952 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 08:35:02.796952 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 08:35:02.796952 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 08:35:02.796952 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 08:35:02.796952 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 08:35:02.796952 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 08:35:02.796952 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 15 08:35:02.796952 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:35:02.796952 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 465/930] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:35:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:35:02.801940 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffe6c33-57cc-4d38-b519-696679ff2dca None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:35:02.907270 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffe6c33-57cc-4d38-b519-696679ff2dca None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:35:02.945802 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3b22b4-1097-405d-a86c-aff03909857e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:35:02.946461 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b22b4-1097-405d-a86c-aff03909857e None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:35:02.946874 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b22b4-1097-405d-a86c-aff03909857e None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:35:02.947205 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b22b4-1097-405d-a86c-aff03909857e None admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:35:02.948175 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3b22b4-1097-405d-a86c-aff03909857e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:35:02.948514 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3b22b4-1097-405d-a86c-aff03909857e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:35:02.950084 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3b22b4-1097-405d-a86c-aff03909857e None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:35:02.969206 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 466/931] 10.222.0.30 () {56 vars in 1143 bytes} [Sat Oct 15 08:35:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:35:02.975322 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2ea83d-e0c5-4570-865a-d8294a05d04d None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:35:03.100310 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2ea83d-e0c5-4570-865a-d8294a05d04d None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:35:03.110982 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:35:03.137323 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86f8c28-3569-4641-82a7-6abae452279c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:35:03.138063 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86f8c28-3569-4641-82a7-6abae452279c None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:35:03.138342 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86f8c28-3569-4641-82a7-6abae452279c None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:35:03.138620 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86f8c28-3569-4641-82a7-6abae452279c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:35:03.213705 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86f8c28-3569-4641-82a7-6abae452279c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:35:03.213705 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86f8c28-3569-4641-82a7-6abae452279c None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:35:03.312777 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 466/932] 10.222.0.30 () {58 vars in 1282 bytes} [Sat Oct 15 08:35:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 15 08:35:03.319791 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3b22b4-1097-405d-a86c-aff03909857e None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:35:03.357834 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:35:03.436394 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3b22b4-1097-405d-a86c-aff03909857e None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:35:03.476562 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7e3299-da73-4643-a218-bab8c01412b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:35:03.477463 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e3299-da73-4643-a218-bab8c01412b0 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:35:03.477890 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e3299-da73-4643-a218-bab8c01412b0 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:35:03.478368 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e3299-da73-4643-a218-bab8c01412b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:35:03.490173 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7e3299-da73-4643-a218-bab8c01412b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:35:03.492749 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7e3299-da73-4643-a218-bab8c01412b0 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:35:03.512284 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-6c7e3299-da73-4643-a218-bab8c01412b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:35:03.512284 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:35:03.512284 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:35:03.512284 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:35:03.512284 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:35:03.512284 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:35:03.512284 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:35:03.512284 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:35:03.512284 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:35:03.512284 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:35:03.512284 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:35:03.512284 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:35:03.512284 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 08:35:03.512284 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 08:35:03.512284 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 08:35:03.512284 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 08:35:03.518116 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:35:03.518116 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:35:03.518116 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:35:03.518116 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:35:03.518116 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:35:03.518116 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:35:03.518116 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:35:03.518116 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:35:03.518116 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:35:03.518116 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:35:03.518116 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:35:03.518116 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:35:03.518116 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:35:03.518116 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:35:03.518116 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:35:03.518116 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:35:03.518116 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 08:35:03.519502 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 08:35:03.519502 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:35:03.519502 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:35:03.519502 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 467/933] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:35:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 08:35:03.519742 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86f8c28-3569-4641-82a7-6abae452279c None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:35:03.531989 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:35:03.639352 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86f8c28-3569-4641-82a7-6abae452279c None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:35:03.675189 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccd361d-b42f-4e26-a8ae-dd674735d0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:35:03.675797 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccd361d-b42f-4e26-a8ae-dd674735d0e7 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:35:03.676076 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccd361d-b42f-4e26-a8ae-dd674735d0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:35:03.676348 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccd361d-b42f-4e26-a8ae-dd674735d0e7 None admin] PATH_INFO: `/v3/domains` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:35:03.677524 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccd361d-b42f-4e26-a8ae-dd674735d0e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:35:03.677867 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccd361d-b42f-4e26-a8ae-dd674735d0e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:35:03.679238 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccd361d-b42f-4e26-a8ae-dd674735d0e7 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:35:03.690905 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 467/934] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 08:35:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:35:03.696791 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7e3299-da73-4643-a218-bab8c01412b0 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:35:03.783100 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:35:03.798938 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7e3299-da73-4643-a218-bab8c01412b0 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:35:03.834578 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37185189-0a0d-4a52-a825-e61b102a22e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:35:03.835312 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37185189-0a0d-4a52-a825-e61b102a22e5 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:35:03.835640 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37185189-0a0d-4a52-a825-e61b102a22e5 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:35:03.835950 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37185189-0a0d-4a52-a825-e61b102a22e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:35:03.843231 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37185189-0a0d-4a52-a825-e61b102a22e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:35:03.844884 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37185189-0a0d-4a52-a825-e61b102a22e5 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:35:03.857090 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-37185189-0a0d-4a52-a825-e61b102a22e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 08:35:03.857090 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:35:03.857090 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:35:03.857090 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:35:03.857090 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:35:03.857090 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:35:03.857090 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:35:03.857090 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:35:03.857090 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:35:03.857090 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:35:03.857090 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:35:03.857090 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:35:03.857090 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 08:35:03.857090 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 08:35:03.857090 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 08:35:03.858237 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 08:35:03.858237 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:35:03.858237 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:35:03.858237 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:35:03.858237 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:35:03.858237 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:35:03.858237 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:35:03.858237 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:35:03.858237 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:35:03.858237 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:35:03.858237 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:35:03.858237 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:35:03.858237 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:35:03.858237 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:35:03.858237 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:35:03.858237 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:35:03.858237 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:35:03.863963 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 08:35:03.863963 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 08:35:03.863963 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 08:35:03.863963 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 08:35:03.863963 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 08:35:03.863963 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 08:35:03.863963 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 08:35:03.863963 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:35:03.863963 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 468/935] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 08:35:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:35:03.864483 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccd361d-b42f-4e26-a8ae-dd674735d0e7 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:35:03.940303 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:35:03.978025 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccd361d-b42f-4e26-a8ae-dd674735d0e7 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:35:04.013980 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222db598-e20c-4bad-8b3a-7c9d9ad249dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:35:04.014578 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222db598-e20c-4bad-8b3a-7c9d9ad249dd None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:35:04.014852 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222db598-e20c-4bad-8b3a-7c9d9ad249dd None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:35:04.015117 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222db598-e20c-4bad-8b3a-7c9d9ad249dd None admin] PATH_INFO: `/v3/projects` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:35:04.015793 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222db598-e20c-4bad-8b3a-7c9d9ad249dd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:35:04.016094 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222db598-e20c-4bad-8b3a-7c9d9ad249dd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:35:04.017502 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222db598-e20c-4bad-8b3a-7c9d9ad249dd None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:35:04.034292 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 468/936] 10.222.0.30 () {56 vars in 1148 bytes} [Sat Oct 15 08:35:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:35:04.041091 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37185189-0a0d-4a52-a825-e61b102a22e5 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:35:04.162491 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37185189-0a0d-4a52-a825-e61b102a22e5 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:35:04.196532 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472ed9c4-4182-4503-9f34-8a52a019a956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:35:04.197354 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ed9c4-4182-4503-9f34-8a52a019a956 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:35:04.197631 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ed9c4-4182-4503-9f34-8a52a019a956 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:35:04.197935 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ed9c4-4182-4503-9f34-8a52a019a956 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:35:04.198659 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472ed9c4-4182-4503-9f34-8a52a019a956 None admin] RBAC: Adding query filter params (role.id=b1b0b29beb794324826a4c4003ae293b, scope.project.id=5dfceb59b3794439a4586b3cbe408f0e, user.id=38986ca2612c4fc59d960cd693834f9b) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:35:04.198969 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472ed9c4-4182-4503-9f34-8a52a019a956 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:35:04.200388 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472ed9c4-4182-4503-9f34-8a52a019a956 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:35:04.206270 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 469/937] 10.222.0.30 () {56 vars in 1374 bytes} [Sat Oct 15 08:35:04 2022] GET /identity/v3/role_assignments?user.id=38986ca2612c4fc59d960cd693834f9b&scope.project.id=5dfceb59b3794439a4586b3cbe408f0e&role.id=b1b0b29beb794324826a4c4003ae293b => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:35:06.483858 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155b2be-ea6a-495d-9dd7-43eb3f032d61 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:35:06.484395 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155b2be-ea6a-495d-9dd7-43eb3f032d61 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:35:06.484549 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155b2be-ea6a-495d-9dd7-43eb3f032d61 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:35:06.486173 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 469/938] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:35:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:35:06.491689 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c915b-6d3b-49dd-b9cb-d35900c79d29 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:35:06.492138 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c915b-6d3b-49dd-b9cb-d35900c79d29 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:35:06.492462 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c915b-6d3b-49dd-b9cb-d35900c79d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:35:06.513315 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-032c915b-6d3b-49dd-b9cb-d35900c79d29 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:35:06.514629 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-032c915b-6d3b-49dd-b9cb-d35900c79d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:35:06.517645 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:35:06.587961 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 470/939] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:35:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 08:35:06.852878 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4155b2be-ea6a-495d-9dd7-43eb3f032d61 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:35:06.965763 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4155b2be-ea6a-495d-9dd7-43eb3f032d61 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:35:06.994179 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:35:07.000800 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45253ba-d783-44fc-a3a1-f01519b00c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:35:07.001418 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45253ba-d783-44fc-a3a1-f01519b00c44 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:35:07.001710 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45253ba-d783-44fc-a3a1-f01519b00c44 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:35:07.001986 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45253ba-d783-44fc-a3a1-f01519b00c44 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:35:07.013402 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45253ba-d783-44fc-a3a1-f01519b00c44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:35:07.018459 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-c45253ba-d783-44fc-a3a1-f01519b00c44 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 08:35:07.018459 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:35:07.018459 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:35:07.018459 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:35:07.018459 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:35:07.018459 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:35:07.018459 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:35:07.018459 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:35:07.018459 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:35:07.018459 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:35:07.018459 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:35:07.018459 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:35:07.018459 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:35:07.018459 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:35:07.018459 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:35:07.018459 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:35:07.019780 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:35:07.019780 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:35:07.019780 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:35:07.019780 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:35:07.019780 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:35:07.019780 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:35:07.019780 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:35:07.019780 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:35:07.019780 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:35:07.019780 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:35:07.019780 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:35:07.019780 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:35:07.019780 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:35:07.019780 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:35:07.019780 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:35:07.019780 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:35:07.020715 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:35:07.020715 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:35:07.020715 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:35:07.020715 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:35:07.020715 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:35:07.020715 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:35:07.020715 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:35:07.020715 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:35:07.020715 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:35:07.020715 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:35:07.020715 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 08:35:07.020715 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:35:07.020715 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 470/940] 10.222.0.30 () {56 vars in 1096 bytes} [Sat Oct 15 08:35:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:35:07.025986 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032c915b-6d3b-49dd-b9cb-d35900c79d29 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:35:07.136237 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032c915b-6d3b-49dd-b9cb-d35900c79d29 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:35:07.171639 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c96557-56de-4cdd-97cd-1a2fb8b4fc20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:35:07.172258 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c96557-56de-4cdd-97cd-1a2fb8b4fc20 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:35:07.172580 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c96557-56de-4cdd-97cd-1a2fb8b4fc20 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:35:07.172928 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c96557-56de-4cdd-97cd-1a2fb8b4fc20 None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:35:07.173596 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c96557-56de-4cdd-97cd-1a2fb8b4fc20 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:35:07.173969 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c96557-56de-4cdd-97cd-1a2fb8b4fc20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:35:07.175398 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c96557-56de-4cdd-97cd-1a2fb8b4fc20 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:35:07.184274 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 471/941] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 08:35:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:35:07.190204 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45253ba-d783-44fc-a3a1-f01519b00c44 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:35:07.305905 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45253ba-d783-44fc-a3a1-f01519b00c44 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:35:07.347409 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af42bd4-92b5-49f0-b46b-9e0814e461b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:35:07.348187 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af42bd4-92b5-49f0-b46b-9e0814e461b0 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:35:07.348863 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af42bd4-92b5-49f0-b46b-9e0814e461b0 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:35:07.349193 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af42bd4-92b5-49f0-b46b-9e0814e461b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:35:07.361318 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af42bd4-92b5-49f0-b46b-9e0814e461b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:35:07.363071 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af42bd4-92b5-49f0-b46b-9e0814e461b0 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:35:07.382439 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-6af42bd4-92b5-49f0-b46b-9e0814e461b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:35:07.382439 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:35:07.382439 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:35:07.382439 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:35:07.382439 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:35:07.382439 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:35:07.382439 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:35:07.382439 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:35:07.382439 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:35:07.382439 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:35:07.382439 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:35:07.382439 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:35:07.382439 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 08:35:07.382439 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 08:35:07.382439 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 08:35:07.382439 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 08:35:07.383468 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:35:07.383468 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:35:07.383468 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:35:07.383468 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:35:07.383468 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:35:07.383468 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:35:07.383468 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:35:07.383468 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:35:07.383468 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:35:07.383468 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:35:07.383468 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:35:07.383468 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:35:07.383468 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:35:07.383468 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:35:07.383468 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:35:07.383468 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:35:07.383468 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 08:35:07.384355 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 08:35:07.384355 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:35:07.384355 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:35:07.384355 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 471/942] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:35:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:35:07.390395 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c96557-56de-4cdd-97cd-1a2fb8b4fc20 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:35:07.393608 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:35:07.496957 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c96557-56de-4cdd-97cd-1a2fb8b4fc20 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:35:07.536628 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7977e12-b261-4de1-8ec4-a2ce3f6fc0aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:35:07.537280 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7977e12-b261-4de1-8ec4-a2ce3f6fc0aa None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:35:07.537578 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7977e12-b261-4de1-8ec4-a2ce3f6fc0aa None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:35:07.537912 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7977e12-b261-4de1-8ec4-a2ce3f6fc0aa None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:35:07.538557 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7977e12-b261-4de1-8ec4-a2ce3f6fc0aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:35:07.538866 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7977e12-b261-4de1-8ec4-a2ce3f6fc0aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:35:07.540317 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7977e12-b261-4de1-8ec4-a2ce3f6fc0aa None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:35:07.554169 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 472/943] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 08:35:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:35:07.561043 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af42bd4-92b5-49f0-b46b-9e0814e461b0 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:35:07.612233 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:35:07.676463 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af42bd4-92b5-49f0-b46b-9e0814e461b0 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:35:07.711317 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2525909d-1f11-41f4-8670-02cfe91aa0d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:35:07.711944 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2525909d-1f11-41f4-8670-02cfe91aa0d9 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:35:07.712235 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2525909d-1f11-41f4-8670-02cfe91aa0d9 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:35:07.712529 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2525909d-1f11-41f4-8670-02cfe91aa0d9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:35:07.719879 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2525909d-1f11-41f4-8670-02cfe91aa0d9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:35:07.721311 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2525909d-1f11-41f4-8670-02cfe91aa0d9 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:35:07.733300 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-2525909d-1f11-41f4-8670-02cfe91aa0d9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 15 08:35:07.733300 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:35:07.733300 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:35:07.733300 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:35:07.733300 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:35:07.733300 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:35:07.733300 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:35:07.733300 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:35:07.733300 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:35:07.733300 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:35:07.733300 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:35:07.733300 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:35:07.733300 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 08:35:07.733300 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 08:35:07.733300 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 08:35:07.733300 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 08:35:07.734532 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:35:07.734532 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:35:07.734532 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 08:35:07.734532 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 08:35:07.734532 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 08:35:07.734532 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 08:35:07.734532 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:35:07.734532 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:35:07.734532 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:35:07.734532 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:35:07.734532 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:35:07.734532 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:35:07.734532 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:35:07.734532 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:35:07.734532 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:35:07.734532 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:35:07.734532 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:35:07.734532 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:35:07.736036 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:35:07.736036 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:35:07.736036 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 08:35:07.736036 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 08:35:07.736036 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 08:35:07.736036 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 08:35:07.736036 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 08:35:07.736036 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 08:35:07.736036 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 15 08:35:07.736036 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:35:07.736036 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 472/944] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:35:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:35:07.740903 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7977e12-b261-4de1-8ec4-a2ce3f6fc0aa None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:35:07.807342 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:35:07.851006 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7977e12-b261-4de1-8ec4-a2ce3f6fc0aa None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:35:07.886776 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff7885c-c5ae-4128-b1fd-c9816fd04b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:35:07.887391 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff7885c-c5ae-4128-b1fd-c9816fd04b82 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:35:07.887683 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff7885c-c5ae-4128-b1fd-c9816fd04b82 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:35:07.887987 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff7885c-c5ae-4128-b1fd-c9816fd04b82 None admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:35:07.888752 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff7885c-c5ae-4128-b1fd-c9816fd04b82 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:35:07.889064 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff7885c-c5ae-4128-b1fd-c9816fd04b82 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:35:07.890524 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff7885c-c5ae-4128-b1fd-c9816fd04b82 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:35:07.909005 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 473/945] 10.222.0.30 () {56 vars in 1143 bytes} [Sat Oct 15 08:35:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:35:07.915549 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2525909d-1f11-41f4-8670-02cfe91aa0d9 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:35:08.032213 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2525909d-1f11-41f4-8670-02cfe91aa0d9 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:35:08.033699 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:35:08.071469 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd85bf0b-12ec-4e0c-bfcf-8710e7602d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:35:08.072092 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85bf0b-12ec-4e0c-bfcf-8710e7602d03 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:35:08.072365 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85bf0b-12ec-4e0c-bfcf-8710e7602d03 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:35:08.072711 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85bf0b-12ec-4e0c-bfcf-8710e7602d03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:35:08.087607 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd85bf0b-12ec-4e0c-bfcf-8710e7602d03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:35:08.089530 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd85bf0b-12ec-4e0c-bfcf-8710e7602d03 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:35:08.109165 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-bd85bf0b-12ec-4e0c-bfcf-8710e7602d03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:35:08.109165 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:35:08.109165 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:35:08.109165 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:35:08.109165 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:35:08.109165 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:35:08.109165 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:35:08.109165 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:35:08.109165 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:35:08.109165 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:35:08.109165 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:35:08.109165 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:35:08.109165 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 08:35:08.109165 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 08:35:08.109165 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 08:35:08.109165 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 08:35:08.110242 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:35:08.110242 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:35:08.110242 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:35:08.110242 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:35:08.110242 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:35:08.110242 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:35:08.110242 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:35:08.110242 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:35:08.110242 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:35:08.110242 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:35:08.110242 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:35:08.110242 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:35:08.110242 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:35:08.110242 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:35:08.110242 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:35:08.110242 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:35:08.110242 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 08:35:08.111133 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 08:35:08.111133 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:35:08.111133 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:35:08.111133 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 473/946] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:35:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 08:35:08.117241 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff7885c-c5ae-4128-b1fd-c9816fd04b82 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:35:08.231491 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff7885c-c5ae-4128-b1fd-c9816fd04b82 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:35:08.253253 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:35:08.269097 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc31839-0e6f-4f1d-aa74-b35eb64630b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:35:08.270087 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc31839-0e6f-4f1d-aa74-b35eb64630b3 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:35:08.270368 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc31839-0e6f-4f1d-aa74-b35eb64630b3 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:35:08.271165 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc31839-0e6f-4f1d-aa74-b35eb64630b3 None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:35:08.271816 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc31839-0e6f-4f1d-aa74-b35eb64630b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:35:08.272134 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc31839-0e6f-4f1d-aa74-b35eb64630b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:35:08.273489 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc31839-0e6f-4f1d-aa74-b35eb64630b3 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:35:08.284734 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 474/947] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 08:35:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:35:08.291000 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd85bf0b-12ec-4e0c-bfcf-8710e7602d03 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:35:08.399714 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd85bf0b-12ec-4e0c-bfcf-8710e7602d03 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:35:08.440235 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b148b7e-b1ca-41c6-a71c-96f7ed485ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:35:08.441570 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b148b7e-b1ca-41c6-a71c-96f7ed485ab2 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:35:08.442208 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b148b7e-b1ca-41c6-a71c-96f7ed485ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:35:08.442723 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b148b7e-b1ca-41c6-a71c-96f7ed485ab2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:35:08.452951 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b148b7e-b1ca-41c6-a71c-96f7ed485ab2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:35:08.455058 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b148b7e-b1ca-41c6-a71c-96f7ed485ab2 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:35:08.471258 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-0b148b7e-b1ca-41c6-a71c-96f7ed485ab2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 08:35:08.471258 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:35:08.471258 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:35:08.471258 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:35:08.471258 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:35:08.471258 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:35:08.471258 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:35:08.471258 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:35:08.471258 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:35:08.471258 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:35:08.471258 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:35:08.471258 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:35:08.471258 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 08:35:08.471258 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 08:35:08.471258 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 08:35:08.472377 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 08:35:08.472377 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:35:08.472377 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:35:08.472377 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:35:08.472377 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:35:08.472377 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:35:08.472377 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:35:08.472377 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:35:08.472377 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:35:08.472377 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:35:08.472377 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:35:08.472377 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:35:08.472377 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:35:08.472377 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:35:08.472377 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:35:08.472377 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:35:08.472377 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:35:08.473476 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 08:35:08.473476 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 08:35:08.473476 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 08:35:08.473476 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 08:35:08.473476 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 08:35:08.473476 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 08:35:08.473476 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 08:35:08.473476 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:35:08.477317 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 474/948] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 08:35:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:35:08.478638 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc31839-0e6f-4f1d-aa74-b35eb64630b3 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:35:08.588118 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc31839-0e6f-4f1d-aa74-b35eb64630b3 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:35:08.623885 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58579648-cbfd-467d-9dde-1010c6ba7186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:35:08.624515 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58579648-cbfd-467d-9dde-1010c6ba7186 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:35:08.624798 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58579648-cbfd-467d-9dde-1010c6ba7186 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:35:08.625072 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58579648-cbfd-467d-9dde-1010c6ba7186 None admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:35:08.625801 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58579648-cbfd-467d-9dde-1010c6ba7186 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:35:08.626110 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58579648-cbfd-467d-9dde-1010c6ba7186 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:35:08.627559 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58579648-cbfd-467d-9dde-1010c6ba7186 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:35:08.641009 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 475/949] 10.222.0.30 () {56 vars in 1148 bytes} [Sat Oct 15 08:35:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:35:08.647815 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b148b7e-b1ca-41c6-a71c-96f7ed485ab2 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:35:08.666374 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:35:08.763063 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b148b7e-b1ca-41c6-a71c-96f7ed485ab2 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:35:08.802694 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63267e49-4b25-4409-82b9-f44503d93fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:35:08.803367 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63267e49-4b25-4409-82b9-f44503d93fd0 None admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:35:08.803649 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63267e49-4b25-4409-82b9-f44503d93fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:35:08.803926 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63267e49-4b25-4409-82b9-f44503d93fd0 None admin] PATH_INFO: `/v3/projects/5dfceb59b3794439a4586b3cbe408f0e/users/38986ca2612c4fc59d960cd693834f9b/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:35:08.806091 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63267e49-4b25-4409-82b9-f44503d93fd0 None admin] RBAC: Authorizing `identity:create_grant(project_id=5dfceb59b3794439a4586b3cbe408f0e, user_id=38986ca2612c4fc59d960cd693834f9b, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:35:08.807774 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63267e49-4b25-4409-82b9-f44503d93fd0 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:35:08.821095 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 475/950] 10.222.0.30 () {58 vars in 1439 bytes} [Sat Oct 15 08:35:08 2022] PUT /identity/v3/projects/5dfceb59b3794439a4586b3cbe408f0e/users/38986ca2612c4fc59d960cd693834f9b/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:35:11.052513 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f12f66-1af7-4a5d-81e1-13cb17fb815f None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:35:11.053082 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f12f66-1af7-4a5d-81e1-13cb17fb815f None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:35:11.053236 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f12f66-1af7-4a5d-81e1-13cb17fb815f None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:35:11.055090 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 476/951] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:35:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:35:11.061343 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0131da1-84c9-47c9-8f85-75d68c0df2fb None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:35:11.061850 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0131da1-84c9-47c9-8f85-75d68c0df2fb None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:35:11.061951 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0131da1-84c9-47c9-8f85-75d68c0df2fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:35:11.081239 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-f0131da1-84c9-47c9-8f85-75d68c0df2fb None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:35:11.082542 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-f0131da1-84c9-47c9-8f85-75d68c0df2fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:35:11.157248 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 476/952] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:35:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 08:35:11.430436 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f12f66-1af7-4a5d-81e1-13cb17fb815f None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:35:11.476127 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:35:11.538314 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f12f66-1af7-4a5d-81e1-13cb17fb815f None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:35:11.573635 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c970db-2c76-47ca-80cd-5e1942ebfd61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:35:11.574320 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c970db-2c76-47ca-80cd-5e1942ebfd61 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:35:11.574607 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c970db-2c76-47ca-80cd-5e1942ebfd61 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:35:11.574889 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c970db-2c76-47ca-80cd-5e1942ebfd61 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:35:11.587069 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c970db-2c76-47ca-80cd-5e1942ebfd61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 08:35:11.592127 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-58c970db-2c76-47ca-80cd-5e1942ebfd61 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 08:35:11.592127 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:35:11.592127 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:35:11.592127 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:35:11.592127 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:35:11.592127 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:35:11.592127 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:35:11.592127 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:35:11.592127 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:35:11.592127 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:35:11.592127 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:35:11.592127 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:35:11.592127 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 08:35:11.592127 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 08:35:11.592127 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 08:35:11.592127 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 08:35:11.593290 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 08:35:11.593290 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 08:35:11.593290 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 08:35:11.593290 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 08:35:11.593290 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:35:11.593290 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:35:11.593290 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:35:11.593290 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:35:11.593290 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:35:11.593290 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:35:11.593290 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:35:11.593290 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:35:11.593290 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:35:11.593290 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:35:11.593290 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:35:11.593290 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:35:11.594183 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:35:11.594183 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:35:11.594183 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:35:11.594183 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:35:11.594183 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 08:35:11.594183 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 08:35:11.594183 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 08:35:11.594183 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 08:35:11.594183 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 08:35:11.594183 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 08:35:11.594183 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 08:35:11.594183 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:35:11.594183 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 477/953] 10.222.0.30 () {56 vars in 1096 bytes} [Sat Oct 15 08:35:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:35:11.599554 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0131da1-84c9-47c9-8f85-75d68c0df2fb None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:35:11.602739 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:35:11.703352 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0131da1-84c9-47c9-8f85-75d68c0df2fb None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:35:11.738381 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e22071-3cd9-47ab-9018-564f753ca346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:35:11.738990 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e22071-3cd9-47ab-9018-564f753ca346 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:35:11.739279 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e22071-3cd9-47ab-9018-564f753ca346 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:35:11.739552 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e22071-3cd9-47ab-9018-564f753ca346 None admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:35:11.740216 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e22071-3cd9-47ab-9018-564f753ca346 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:35:11.740520 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e22071-3cd9-47ab-9018-564f753ca346 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:35:11.741926 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e22071-3cd9-47ab-9018-564f753ca346 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:35:11.750009 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 477/954] 10.222.0.30 () {56 vars in 1099 bytes} [Sat Oct 15 08:35:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:35:11.756396 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c970db-2c76-47ca-80cd-5e1942ebfd61 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:35:11.856386 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:35:11.859791 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c970db-2c76-47ca-80cd-5e1942ebfd61 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:35:11.893899 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459f879a-bf8f-4357-a27d-36d62ca6c3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:35:11.894792 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f879a-bf8f-4357-a27d-36d62ca6c3c6 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:35:11.895077 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f879a-bf8f-4357-a27d-36d62ca6c3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:35:11.895357 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f879a-bf8f-4357-a27d-36d62ca6c3c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:35:11.905743 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459f879a-bf8f-4357-a27d-36d62ca6c3c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:35:11.907174 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459f879a-bf8f-4357-a27d-36d62ca6c3c6 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:35:11.922502 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-459f879a-bf8f-4357-a27d-36d62ca6c3c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:35:11.922502 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:35:11.922502 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:35:11.922502 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:35:11.922502 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:35:11.922502 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:35:11.922502 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:35:11.922502 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:35:11.922502 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:35:11.922502 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:35:11.922502 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:35:11.922502 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:35:11.922502 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 08:35:11.922502 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 08:35:11.922502 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 08:35:11.922502 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 08:35:11.923529 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:35:11.923529 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:35:11.923529 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:35:11.923529 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:35:11.923529 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:35:11.923529 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:35:11.923529 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:35:11.923529 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:35:11.923529 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:35:11.923529 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:35:11.923529 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:35:11.923529 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:35:11.923529 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:35:11.923529 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:35:11.923529 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:35:11.923529 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:35:11.923529 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 08:35:11.924868 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 08:35:11.924868 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:35:11.924868 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:35:11.924868 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 478/955] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:35:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:35:11.930090 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e22071-3cd9-47ab-9018-564f753ca346 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:35:11.997317 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:35:12.032482 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e22071-3cd9-47ab-9018-564f753ca346 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:35:12.069148 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2869549c-9b34-4851-b06f-d3bf492c9875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:35:12.070018 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2869549c-9b34-4851-b06f-d3bf492c9875 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:35:12.070167 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2869549c-9b34-4851-b06f-d3bf492c9875 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:35:12.070358 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2869549c-9b34-4851-b06f-d3bf492c9875 None admin] PATH_INFO: `/v3/domains` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:35:12.071047 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2869549c-9b34-4851-b06f-d3bf492c9875 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:35:12.071394 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2869549c-9b34-4851-b06f-d3bf492c9875 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:35:12.072844 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2869549c-9b34-4851-b06f-d3bf492c9875 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:35:12.086764 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 478/956] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 08:35:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:35:12.092796 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459f879a-bf8f-4357-a27d-36d62ca6c3c6 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:35:12.189291 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459f879a-bf8f-4357-a27d-36d62ca6c3c6 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:35:12.221144 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107a0f6b-fd34-46e2-bd99-f8928fc27364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:35:12.221785 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a0f6b-fd34-46e2-bd99-f8928fc27364 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:35:12.222067 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a0f6b-fd34-46e2-bd99-f8928fc27364 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:35:12.222340 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a0f6b-fd34-46e2-bd99-f8928fc27364 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:35:12.229953 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107a0f6b-fd34-46e2-bd99-f8928fc27364 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:35:12.231373 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107a0f6b-fd34-46e2-bd99-f8928fc27364 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:35:12.242097 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-107a0f6b-fd34-46e2-bd99-f8928fc27364 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 15 08:35:12.242097 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:35:12.242097 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:35:12.242097 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:35:12.242097 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:35:12.242097 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:35:12.242097 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:35:12.242097 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:35:12.242097 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:35:12.242097 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:35:12.242097 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:35:12.242097 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:35:12.242097 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 08:35:12.242097 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 08:35:12.242097 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 08:35:12.242097 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 08:35:12.243238 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:35:12.243238 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:35:12.243238 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 08:35:12.243238 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 08:35:12.243238 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 08:35:12.243238 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 08:35:12.243238 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:35:12.243238 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:35:12.243238 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:35:12.243238 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:35:12.243238 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:35:12.243238 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:35:12.243238 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:35:12.243238 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:35:12.243238 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:35:12.243238 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:35:12.243238 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:35:12.243238 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:35:12.244407 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:35:12.244407 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:35:12.244407 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 08:35:12.244407 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 08:35:12.244407 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 08:35:12.244407 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 08:35:12.244407 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 08:35:12.244407 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 08:35:12.244407 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 15 08:35:12.244407 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:35:12.244407 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 479/957] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:35:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:35:12.248973 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2869549c-9b34-4851-b06f-d3bf492c9875 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:35:12.344001 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2869549c-9b34-4851-b06f-d3bf492c9875 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:35:12.367806 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:35:12.376780 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da037c5-f998-4c09-aae1-e932474b4d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:35:12.377382 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da037c5-f998-4c09-aae1-e932474b4d62 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:35:12.377663 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da037c5-f998-4c09-aae1-e932474b4d62 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:35:12.377975 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da037c5-f998-4c09-aae1-e932474b4d62 None admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:35:12.378690 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da037c5-f998-4c09-aae1-e932474b4d62 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:35:12.378996 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da037c5-f998-4c09-aae1-e932474b4d62 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:35:12.380964 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da037c5-f998-4c09-aae1-e932474b4d62 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:35:12.398497 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 479/958] 10.222.0.30 () {56 vars in 1143 bytes} [Sat Oct 15 08:35:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:35:12.404159 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107a0f6b-fd34-46e2-bd99-f8928fc27364 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:35:12.419823 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:35:12.501268 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107a0f6b-fd34-46e2-bd99-f8928fc27364 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:35:12.532883 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6ab364-0243-4b93-a94b-41b66ddb2028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:35:12.533476 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ab364-0243-4b93-a94b-41b66ddb2028 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:35:12.533788 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ab364-0243-4b93-a94b-41b66ddb2028 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:35:12.534069 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ab364-0243-4b93-a94b-41b66ddb2028 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:35:12.597440 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6ab364-0243-4b93-a94b-41b66ddb2028 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:35:12.599396 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6ab364-0243-4b93-a94b-41b66ddb2028 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:35:12.612001 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:35:12.692454 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 480/959] 10.222.0.30 () {58 vars in 1282 bytes} [Sat Oct 15 08:35:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 15 08:35:12.698180 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da037c5-f998-4c09-aae1-e932474b4d62 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:35:12.792320 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da037c5-f998-4c09-aae1-e932474b4d62 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:35:12.824958 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65486f92-b6b1-4b18-aab5-f2f4fc6d1311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:35:12.825563 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65486f92-b6b1-4b18-aab5-f2f4fc6d1311 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:35:12.825936 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65486f92-b6b1-4b18-aab5-f2f4fc6d1311 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:35:12.826210 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65486f92-b6b1-4b18-aab5-f2f4fc6d1311 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:35:12.835798 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65486f92-b6b1-4b18-aab5-f2f4fc6d1311 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:35:12.837221 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65486f92-b6b1-4b18-aab5-f2f4fc6d1311 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:35:12.851107 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-65486f92-b6b1-4b18-aab5-f2f4fc6d1311 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:35:12.851107 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:35:12.851107 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:35:12.851107 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:35:12.851107 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:35:12.851107 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:35:12.851107 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:35:12.851107 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:35:12.851107 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:35:12.851107 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:35:12.851107 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:35:12.851107 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:35:12.851107 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 08:35:12.851107 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 08:35:12.851107 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 08:35:12.851107 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 08:35:12.852204 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:35:12.852204 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:35:12.852204 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:35:12.852204 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:35:12.852204 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:35:12.852204 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:35:12.852204 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:35:12.852204 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:35:12.852204 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:35:12.852204 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:35:12.852204 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:35:12.852204 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:35:12.852204 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:35:12.852204 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:35:12.852204 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:35:12.852204 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:35:12.852204 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 08:35:12.853104 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 08:35:12.853104 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 08:35:12.853104 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:35:12.853104 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 480/960] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:35:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:35:12.857655 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6ab364-0243-4b93-a94b-41b66ddb2028 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:35:12.952996 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6ab364-0243-4b93-a94b-41b66ddb2028 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:35:12.966606 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:35:12.987187 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774e22ae-04bf-4e2c-8252-2b91872cad35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:35:12.987788 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e22ae-04bf-4e2c-8252-2b91872cad35 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:35:12.988070 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e22ae-04bf-4e2c-8252-2b91872cad35 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:35:12.988344 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e22ae-04bf-4e2c-8252-2b91872cad35 None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:35:12.988975 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774e22ae-04bf-4e2c-8252-2b91872cad35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:35:12.989844 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774e22ae-04bf-4e2c-8252-2b91872cad35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:35:12.991191 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774e22ae-04bf-4e2c-8252-2b91872cad35 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:35:13.002084 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 481/961] 10.222.0.30 () {56 vars in 1109 bytes} [Sat Oct 15 08:35:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:35:13.008203 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65486f92-b6b1-4b18-aab5-f2f4fc6d1311 None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:35:13.041996 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:35:13.111091 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65486f92-b6b1-4b18-aab5-f2f4fc6d1311 None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:35:13.143217 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b276d-6510-46d1-a81a-51342d5e3a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:35:13.143857 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b276d-6510-46d1-a81a-51342d5e3a6a None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:35:13.144136 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b276d-6510-46d1-a81a-51342d5e3a6a None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:35:13.144665 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b276d-6510-46d1-a81a-51342d5e3a6a None admin] PATH_INFO: `/v3/projects/service` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:35:13.150988 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475b276d-6510-46d1-a81a-51342d5e3a6a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:35:13.152405 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475b276d-6510-46d1-a81a-51342d5e3a6a None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:35:13.163551 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-475b276d-6510-46d1-a81a-51342d5e3a6a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 08:35:13.163551 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:35:13.163551 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:35:13.163551 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:35:13.163551 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:35:13.163551 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:35:13.163551 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:35:13.163551 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:35:13.163551 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:35:13.163551 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:35:13.163551 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:35:13.163551 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:35:13.163551 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 08:35:13.163551 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 08:35:13.163551 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 08:35:13.164578 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 08:35:13.164578 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:35:13.164578 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:35:13.164578 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:35:13.164578 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:35:13.164578 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:35:13.164578 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:35:13.164578 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:35:13.164578 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 08:35:13.164578 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:35:13.164578 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:35:13.164578 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:35:13.164578 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:35:13.164578 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:35:13.164578 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:35:13.164578 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:35:13.164578 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:35:13.165724 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 08:35:13.165724 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 08:35:13.165724 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 08:35:13.165724 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 08:35:13.165724 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 08:35:13.165724 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 08:35:13.165724 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 08:35:13.165724 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 08:35:13.165724 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 481/962] 10.222.0.30 () {56 vars in 1111 bytes} [Sat Oct 15 08:35:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:35:13.170503 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774e22ae-04bf-4e2c-8252-2b91872cad35 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:35:13.265595 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774e22ae-04bf-4e2c-8252-2b91872cad35 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:35:13.297650 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b3bd5f-b135-4a24-98c2-acdbeb941bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:35:13.298265 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b3bd5f-b135-4a24-98c2-acdbeb941bd0 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:35:13.298549 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b3bd5f-b135-4a24-98c2-acdbeb941bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:35:13.298825 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b3bd5f-b135-4a24-98c2-acdbeb941bd0 None admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:35:13.299501 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b3bd5f-b135-4a24-98c2-acdbeb941bd0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:35:13.299805 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b3bd5f-b135-4a24-98c2-acdbeb941bd0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:35:13.301206 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b3bd5f-b135-4a24-98c2-acdbeb941bd0 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:35:13.312619 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 482/963] 10.222.0.30 () {56 vars in 1148 bytes} [Sat Oct 15 08:35:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:35:13.318894 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b276d-6510-46d1-a81a-51342d5e3a6a None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:35:13.409485 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:35:13.415812 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b276d-6510-46d1-a81a-51342d5e3a6a None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:35:13.448267 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126ace65-964a-4033-bff0-1b37d8b060be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:35:13.448864 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ace65-964a-4033-bff0-1b37d8b060be None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:35:13.449135 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ace65-964a-4033-bff0-1b37d8b060be None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:35:13.449403 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ace65-964a-4033-bff0-1b37d8b060be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:35:13.450142 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126ace65-964a-4033-bff0-1b37d8b060be None admin] RBAC: Adding query filter params (role.id=14615e30ed3f4d8cb78ee5e0bf9b5761, scope.project.id=5dfceb59b3794439a4586b3cbe408f0e, user.id=38986ca2612c4fc59d960cd693834f9b) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:35:13.450449 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126ace65-964a-4033-bff0-1b37d8b060be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:35:13.451866 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126ace65-964a-4033-bff0-1b37d8b060be None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:35:13.457405 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 482/964] 10.222.0.30 () {56 vars in 1374 bytes} [Sat Oct 15 08:35:13 2022] GET /identity/v3/role_assignments?user.id=38986ca2612c4fc59d960cd693834f9b&scope.project.id=5dfceb59b3794439a4586b3cbe408f0e&role.id=14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:35:15.581178 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879f521a-591b-466e-b3b0-e85201067939 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:35:15.581740 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879f521a-591b-466e-b3b0-e85201067939 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:35:15.581835 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879f521a-591b-466e-b3b0-e85201067939 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:35:15.583762 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 483/965] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:35:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:35:15.588823 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51dc407-195b-4918-9a3e-808244838fb5 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:35:15.589246 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51dc407-195b-4918-9a3e-808244838fb5 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:35:15.589395 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51dc407-195b-4918-9a3e-808244838fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:35:15.607083 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-c51dc407-195b-4918-9a3e-808244838fb5 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:35:15.608247 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-c51dc407-195b-4918-9a3e-808244838fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:35:15.678610 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 483/966] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:35:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 08:35:15.943071 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879f521a-591b-466e-b3b0-e85201067939 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:35:15.974261 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:35:16.042110 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879f521a-591b-466e-b3b0-e85201067939 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:35:16.077223 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d53da98-941d-437d-951d-2288d8f129ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:35:16.077960 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53da98-941d-437d-951d-2288d8f129ab None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:35:16.078284 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53da98-941d-437d-951d-2288d8f129ab None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:35:16.078603 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53da98-941d-437d-951d-2288d8f129ab None admin] PATH_INFO: `/v3/services/placement` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:35:16.079240 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d53da98-941d-437d-951d-2288d8f129ab None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:35:16.080788 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d53da98-941d-437d-951d-2288d8f129ab None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:35:16.091301 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-8d53da98-941d-437d-951d-2288d8f129ab None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 15 08:35:16.091301 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:35:16.091301 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 08:35:16.091301 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 08:35:16.091301 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 08:35:16.091301 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 08:35:16.091301 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 08:35:16.091301 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 08:35:16.091301 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 15 08:35:16.091301 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:35:16.091301 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 08:35:16.091301 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:35:16.091301 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:35:16.091301 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:35:16.091301 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:35:16.091301 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:35:16.091301 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:35:16.092989 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:35:16.092989 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:35:16.092989 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:35:16.092989 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:35:16.092989 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:35:16.092989 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:35:16.092989 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 08:35:16.092989 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 08:35:16.092989 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 08:35:16.092989 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 08:35:16.092989 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:35:16.092989 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:35:16.092989 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:35:16.092989 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:35:16.092989 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:35:16.092989 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:35:16.094277 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:35:16.094277 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:35:16.094277 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:35:16.094277 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:35:16.094277 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:35:16.094277 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:35:16.094277 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:35:16.094277 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:35:16.094277 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:35:16.094277 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:35:16.094277 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 08:35:16.094277 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 08:35:16.094277 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 15 08:35:16.094277 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:35:16.094277 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 484/967] 10.222.0.30 () {56 vars in 1117 bytes} [Sat Oct 15 08:35:15 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:35:16.098688 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51dc407-195b-4918-9a3e-808244838fb5 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:35:16.188022 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:35:16.197916 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51dc407-195b-4918-9a3e-808244838fb5 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:35:16.230468 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cbba9b-9fbd-4251-8fab-2f761e0fcbab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:35:16.231073 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cbba9b-9fbd-4251-8fab-2f761e0fcbab None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:35:16.231356 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cbba9b-9fbd-4251-8fab-2f761e0fcbab None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:35:16.231632 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cbba9b-9fbd-4251-8fab-2f761e0fcbab None admin] PATH_INFO: `/v3/services` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:35:16.232252 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cbba9b-9fbd-4251-8fab-2f761e0fcbab None admin] RBAC: Adding query filter params (name=placement) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:35:16.232559 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cbba9b-9fbd-4251-8fab-2f761e0fcbab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:35:16.233959 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cbba9b-9fbd-4251-8fab-2f761e0fcbab None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:35:16.238605 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 484/968] 10.222.0.30 () {56 vars in 1116 bytes} [Sat Oct 15 08:35:16 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:35:16.244392 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d53da98-941d-437d-951d-2288d8f129ab None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:35:16.337942 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:35:16.343755 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d53da98-941d-437d-951d-2288d8f129ab None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:35:16.375934 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abee6e0-9306-49e3-a2f2-dfdc911f40b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:35:16.376528 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abee6e0-9306-49e3-a2f2-dfdc911f40b7 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:35:16.376810 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abee6e0-9306-49e3-a2f2-dfdc911f40b7 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:35:16.377086 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abee6e0-9306-49e3-a2f2-dfdc911f40b7 None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:35:16.377737 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abee6e0-9306-49e3-a2f2-dfdc911f40b7 None admin] RBAC: Adding query filter params (type=placement) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:35:16.378048 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abee6e0-9306-49e3-a2f2-dfdc911f40b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:35:16.379400 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abee6e0-9306-49e3-a2f2-dfdc911f40b7 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:35:16.384337 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 485/969] 10.222.0.30 () {56 vars in 1116 bytes} [Sat Oct 15 08:35:16 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:35:18.491213 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dfd1e3-afa6-449e-8a00-463452a1c83b None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:35:18.491748 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dfd1e3-afa6-449e-8a00-463452a1c83b None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:35:18.491748 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dfd1e3-afa6-449e-8a00-463452a1c83b None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:35:18.493098 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 485/970] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:35:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:35:18.498176 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2116b165-183d-47e3-9be1-431379e760b7 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:35:18.498600 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2116b165-183d-47e3-9be1-431379e760b7 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:35:18.498728 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2116b165-183d-47e3-9be1-431379e760b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:35:18.516742 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-2116b165-183d-47e3-9be1-431379e760b7 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:35:18.517943 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-2116b165-183d-47e3-9be1-431379e760b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:35:18.581623 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 486/971] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:35:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 08:35:18.851171 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dfd1e3-afa6-449e-8a00-463452a1c83b None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:35:18.950121 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dfd1e3-afa6-449e-8a00-463452a1c83b None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:35:18.982338 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e059f0b-87aa-4c6c-b0a1-25ac00d04f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:35:18.982926 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e059f0b-87aa-4c6c-b0a1-25ac00d04f7a None admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:35:18.983207 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e059f0b-87aa-4c6c-b0a1-25ac00d04f7a None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:35:18.983479 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e059f0b-87aa-4c6c-b0a1-25ac00d04f7a None admin] PATH_INFO: `/v3/services` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:35:18.984128 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e059f0b-87aa-4c6c-b0a1-25ac00d04f7a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:35:18.985481 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e059f0b-87aa-4c6c-b0a1-25ac00d04f7a None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:35:19.000170 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 486/972] 10.222.0.30 () {60 vars in 1141 bytes} [Sat Oct 15 08:35:18 2022] POST /identity/v3/services => generated 246 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:35:21.128016 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b8414-8832-4c38-8f31-55660206a2ed None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:35:21.128563 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b8414-8832-4c38-8f31-55660206a2ed None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:35:21.128563 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b8414-8832-4c38-8f31-55660206a2ed None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:35:21.130528 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 487/973] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:35:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:35:21.136612 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8080bc4d-5059-42a8-a92d-c5c2b6f77787 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:35:21.137113 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8080bc4d-5059-42a8-a92d-c5c2b6f77787 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:35:21.137264 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8080bc4d-5059-42a8-a92d-c5c2b6f77787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:35:21.158645 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-8080bc4d-5059-42a8-a92d-c5c2b6f77787 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:35:21.159983 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-8080bc4d-5059-42a8-a92d-c5c2b6f77787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:35:21.171500 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:35:21.234057 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 487/974] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:35:21 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 08:35:21.502899 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040b8414-8832-4c38-8f31-55660206a2ed None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:35:21.589462 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:35:21.600212 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040b8414-8832-4c38-8f31-55660206a2ed None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:35:21.632751 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ad5031-359c-40f8-b798-ec4a054d6be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:35:21.633365 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad5031-359c-40f8-b798-ec4a054d6be4 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:35:21.633646 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad5031-359c-40f8-b798-ec4a054d6be4 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:35:21.633956 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad5031-359c-40f8-b798-ec4a054d6be4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:35:21.634529 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ad5031-359c-40f8-b798-ec4a054d6be4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:35:21.635871 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ad5031-359c-40f8-b798-ec4a054d6be4 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:35:21.644398 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-b0ad5031-359c-40f8-b798-ec4a054d6be4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 15 08:35:21.644398 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:35:21.644398 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 08:35:21.644398 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 08:35:21.644398 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 08:35:21.644398 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 08:35:21.644398 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 08:35:21.644398 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 08:35:21.644398 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 15 08:35:21.644398 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:35:21.644398 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 08:35:21.644398 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:35:21.644398 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:35:21.644398 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:35:21.644398 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:35:21.644398 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:35:21.644398 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:35:21.645962 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:35:21.645962 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:35:21.645962 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:35:21.645962 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:35:21.645962 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:35:21.645962 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:35:21.645962 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 08:35:21.645962 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 08:35:21.645962 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 08:35:21.645962 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 08:35:21.645962 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:35:21.645962 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:35:21.645962 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:35:21.645962 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:35:21.645962 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:35:21.645962 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:35:21.646906 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:35:21.646906 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:35:21.646906 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:35:21.646906 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:35:21.646906 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:35:21.646906 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:35:21.646906 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:35:21.646906 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:35:21.646906 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:35:21.646906 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:35:21.646906 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 08:35:21.646906 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 08:35:21.646906 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 15 08:35:21.646906 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:35:21.646906 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 488/975] 10.222.0.30 () {56 vars in 1117 bytes} [Sat Oct 15 08:35:21 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:35:21.651733 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8080bc4d-5059-42a8-a92d-c5c2b6f77787 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:35:21.752227 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8080bc4d-5059-42a8-a92d-c5c2b6f77787 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:35:21.784751 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e31870-e7c9-4671-97aa-b2765e79472a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:35:21.785338 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e31870-e7c9-4671-97aa-b2765e79472a None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:35:21.785613 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e31870-e7c9-4671-97aa-b2765e79472a None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:35:21.785911 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e31870-e7c9-4671-97aa-b2765e79472a None admin] PATH_INFO: `/v3/services` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:35:21.786528 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e31870-e7c9-4671-97aa-b2765e79472a None admin] RBAC: Adding query filter params (name=placement) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:35:21.786827 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e31870-e7c9-4671-97aa-b2765e79472a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:35:21.788181 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e31870-e7c9-4671-97aa-b2765e79472a None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:35:21.793056 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 488/976] 10.222.0.30 () {56 vars in 1116 bytes} [Sat Oct 15 08:35:21 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:35:21.798946 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ad5031-359c-40f8-b798-ec4a054d6be4 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:35:21.895447 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ad5031-359c-40f8-b798-ec4a054d6be4 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:35:21.927275 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97231a20-71db-462a-a0a4-dd8b5f7973d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:35:21.927796 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97231a20-71db-462a-a0a4-dd8b5f7973d3 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:35:21.928076 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97231a20-71db-462a-a0a4-dd8b5f7973d3 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:35:21.928351 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97231a20-71db-462a-a0a4-dd8b5f7973d3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:35:21.929038 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97231a20-71db-462a-a0a4-dd8b5f7973d3 None admin] RBAC: Adding query filter params (interface=public, service_id=6ff2817b180d419985450b23becc3704, region_id=RegionOne) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:35:21.929344 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97231a20-71db-462a-a0a4-dd8b5f7973d3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:35:21.930720 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97231a20-71db-462a-a0a4-dd8b5f7973d3 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:35:21.935492 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 489/977] 10.222.0.30 () {56 vars in 1251 bytes} [Sat Oct 15 08:35:21 2022] GET /identity/v3/endpoints?service_id=6ff2817b180d419985450b23becc3704&interface=public®ion_id=RegionOne => generated 194 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:35:21.940949 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e31870-e7c9-4671-97aa-b2765e79472a None admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:35:21.944661 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:35:22.038310 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e31870-e7c9-4671-97aa-b2765e79472a None admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:35:22.073929 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7f3f09-cc25-4bba-a6cb-d44ec9ba4cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:35:22.074526 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f3f09-cc25-4bba-a6cb-d44ec9ba4cf4 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:35:22.074805 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f3f09-cc25-4bba-a6cb-d44ec9ba4cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:35:22.075076 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f3f09-cc25-4bba-a6cb-d44ec9ba4cf4 None admin] PATH_INFO: `/v3/services` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:35:22.075670 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7f3f09-cc25-4bba-a6cb-d44ec9ba4cf4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:35:22.077050 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7f3f09-cc25-4bba-a6cb-d44ec9ba4cf4 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:35:22.084456 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 489/978] 10.222.0.30 () {56 vars in 1087 bytes} [Sat Oct 15 08:35:21 2022] GET /identity/v3/services => generated 2191 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:35:24.190061 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71113f29-1bb6-41b9-9b15-8232318e374a None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:35:24.190614 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71113f29-1bb6-41b9-9b15-8232318e374a None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:35:24.190614 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71113f29-1bb6-41b9-9b15-8232318e374a None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:35:24.191945 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 490/979] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:35:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:35:24.197643 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc63dc7-b075-4646-8d21-ebfcaf03d11e None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:35:24.198053 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc63dc7-b075-4646-8d21-ebfcaf03d11e None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:35:24.198214 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc63dc7-b075-4646-8d21-ebfcaf03d11e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:35:24.216245 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-7bc63dc7-b075-4646-8d21-ebfcaf03d11e None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:35:24.217415 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-7bc63dc7-b075-4646-8d21-ebfcaf03d11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:35:24.276657 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:35:24.282215 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 490/980] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:35:24 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 08:35:24.549911 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71113f29-1bb6-41b9-9b15-8232318e374a None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:35:24.569528 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:35:24.648439 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71113f29-1bb6-41b9-9b15-8232318e374a None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:35:24.680999 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d842ce9b-66e7-419b-8ad3-4b981d7a1c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:35:24.681606 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d842ce9b-66e7-419b-8ad3-4b981d7a1c4e None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:35:24.681911 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d842ce9b-66e7-419b-8ad3-4b981d7a1c4e None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:35:24.682861 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d842ce9b-66e7-419b-8ad3-4b981d7a1c4e None admin] PATH_INFO: `/v3/services/placement` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:35:24.683520 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d842ce9b-66e7-419b-8ad3-4b981d7a1c4e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:35:24.684877 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d842ce9b-66e7-419b-8ad3-4b981d7a1c4e None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:35:24.693848 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-d842ce9b-66e7-419b-8ad3-4b981d7a1c4e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 15 08:35:24.693848 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:35:24.693848 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 08:35:24.693848 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 08:35:24.693848 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 08:35:24.693848 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 08:35:24.693848 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 08:35:24.693848 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 08:35:24.693848 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 15 08:35:24.693848 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:35:24.693848 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 08:35:24.693848 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:35:24.693848 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 08:35:24.693848 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 08:35:24.693848 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 08:35:24.693848 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 08:35:24.693848 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 08:35:24.695054 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 08:35:24.695054 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 08:35:24.695054 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 08:35:24.695054 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 08:35:24.695054 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 08:35:24.695054 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 08:35:24.695054 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 08:35:24.695054 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 08:35:24.695054 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 08:35:24.695054 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 08:35:24.695054 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 08:35:24.695054 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 08:35:24.695054 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 08:35:24.695054 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 08:35:24.695054 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 08:35:24.695054 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 08:35:24.696115 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 08:35:24.696115 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 08:35:24.696115 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 08:35:24.696115 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 08:35:24.696115 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 08:35:24.696115 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 08:35:24.696115 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 08:35:24.696115 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 08:35:24.696115 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 08:35:24.696115 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 08:35:24.696115 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 08:35:24.696115 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 08:35:24.696115 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 15 08:35:24.696115 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 08:35:24.696115 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 491/981] 10.222.0.30 () {56 vars in 1117 bytes} [Sat Oct 15 08:35:24 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 08:35:24.700861 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc63dc7-b075-4646-8d21-ebfcaf03d11e None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:35:24.796406 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc63dc7-b075-4646-8d21-ebfcaf03d11e None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:35:24.829168 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da92653c-e7b8-4ff8-8302-36d08cb51312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:35:24.829724 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da92653c-e7b8-4ff8-8302-36d08cb51312 None admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:35:24.830012 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da92653c-e7b8-4ff8-8302-36d08cb51312 None admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:35:24.830286 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da92653c-e7b8-4ff8-8302-36d08cb51312 None admin] PATH_INFO: `/v3/services` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:35:24.830906 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da92653c-e7b8-4ff8-8302-36d08cb51312 None admin] RBAC: Adding query filter params (name=placement) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:35:24.831211 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da92653c-e7b8-4ff8-8302-36d08cb51312 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:35:24.832579 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da92653c-e7b8-4ff8-8302-36d08cb51312 None admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:35:24.838043 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 491/982] 10.222.0.30 () {56 vars in 1116 bytes} [Sat Oct 15 08:35:24 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:35:24.843750 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d842ce9b-66e7-419b-8ad3-4b981d7a1c4e None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:35:24.922171 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:35:24.940020 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d842ce9b-66e7-419b-8ad3-4b981d7a1c4e None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:35:24.971735 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f99378e-a2e0-4e0f-9eac-f40e561ba5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:35:24.972324 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f99378e-a2e0-4e0f-9eac-f40e561ba5b0 None admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:35:24.972604 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f99378e-a2e0-4e0f-9eac-f40e561ba5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:35:24.972874 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f99378e-a2e0-4e0f-9eac-f40e561ba5b0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:35:24.974187 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f99378e-a2e0-4e0f-9eac-f40e561ba5b0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:35:24.975528 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f99378e-a2e0-4e0f-9eac-f40e561ba5b0 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:35:24.993684 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 492/983] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:35:24 2022] POST /identity/v3/endpoints => generated 331 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 08:35:34.835100 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=69 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:35:34.989993 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:36:28.345564 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e087d859-9afd-414e-a12a-c656cf679eca None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:36:28.346042 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e087d859-9afd-414e-a12a-c656cf679eca None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:36:28.346214 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e087d859-9afd-414e-a12a-c656cf679eca None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:36:28.348304 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 492/984] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:36:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:36:28.353652 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42733f-549e-4a4d-ba99-440dee53a8df None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:36:28.354119 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42733f-549e-4a4d-ba99-440dee53a8df None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:36:28.354515 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42733f-549e-4a4d-ba99-440dee53a8df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:36:28.467557 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-7e42733f-549e-4a4d-ba99-440dee53a8df None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:36:28.468894 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-7e42733f-549e-4a4d-ba99-440dee53a8df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:36:28.581258 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 493/985] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:36:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:36:28.875178 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a83329e-df08-4354-abc9-7119044a03e5 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:36:28.875536 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a83329e-df08-4354-abc9-7119044a03e5 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:36:28.876016 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a83329e-df08-4354-abc9-7119044a03e5 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:36:28.877455 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 493/986] 10.222.0.30 () {54 vars in 964 bytes} [Sat Oct 15 08:36:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:36:28.884397 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d8d5c1-d101-4903-804c-6db30e4f7f6b None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:36:28.884732 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d8d5c1-d101-4903-804c-6db30e4f7f6b None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:36:28.885099 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d8d5c1-d101-4903-804c-6db30e4f7f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:36:28.971359 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-e8d8d5c1-d101-4903-804c-6db30e4f7f6b None None] MFA Rules not processed for user `8b11e349e4cb4558a4c75d843effc0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:36:28.973101 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-e8d8d5c1-d101-4903-804c-6db30e4f7f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:36:28.994780 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:36:29.055990 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 494/987] 10.222.0.30 () {58 vars in 1061 bytes} [Sat Oct 15 08:36:28 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:36:29.065086 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a83329e-df08-4354-abc9-7119044a03e5 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:36:29.067471 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-2a83329e-df08-4354-abc9-7119044a03e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:36:29.177754 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a83329e-df08-4354-abc9-7119044a03e5 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:36:29.253938 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a37ef0-cc6f-4ed5-8345-a31b66655838 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:36:29.254553 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a37ef0-cc6f-4ed5-8345-a31b66655838 service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:36:29.254831 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a37ef0-cc6f-4ed5-8345-a31b66655838 service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:36:29.255180 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a37ef0-cc6f-4ed5-8345-a31b66655838 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:36:29.256679 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-16a37ef0-cc6f-4ed5-8345-a31b66655838 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:36:29.314156 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:36:29.372119 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a37ef0-cc6f-4ed5-8345-a31b66655838 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:36:29.373566 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a37ef0-cc6f-4ed5-8345-a31b66655838 service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:36:29.399195 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 494/988] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:36:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:36:31.841998 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3910bc6b-09eb-4dba-b784-8036b82971d0 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:36:31.842504 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3910bc6b-09eb-4dba-b784-8036b82971d0 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:36:31.842504 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3910bc6b-09eb-4dba-b784-8036b82971d0 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:36:31.843820 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 495/989] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:36:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:36:31.849550 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db55af-6e9d-448a-924d-4b62135e4798 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:36:31.850014 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db55af-6e9d-448a-924d-4b62135e4798 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:36:31.850275 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db55af-6e9d-448a-924d-4b62135e4798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:36:31.922038 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-31db55af-6e9d-448a-924d-4b62135e4798 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:36:31.923257 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-31db55af-6e9d-448a-924d-4b62135e4798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:36:32.018315 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 495/990] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:36:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:36:32.307306 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3910bc6b-09eb-4dba-b784-8036b82971d0 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:36:32.308832 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-3910bc6b-09eb-4dba-b784-8036b82971d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:36:32.419835 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3910bc6b-09eb-4dba-b784-8036b82971d0 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:36:32.497140 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:36:32.499961 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a047e4-eb7f-4c32-bf4e-0260c1757b60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:36:32.500994 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a047e4-eb7f-4c32-bf4e-0260c1757b60 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:36:32.501490 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a047e4-eb7f-4c32-bf4e-0260c1757b60 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:36:32.502069 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a047e4-eb7f-4c32-bf4e-0260c1757b60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:36:32.504171 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-b0a047e4-eb7f-4c32-bf4e-0260c1757b60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:36:32.615485 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a047e4-eb7f-4c32-bf4e-0260c1757b60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:36:32.616841 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a047e4-eb7f-4c32-bf4e-0260c1757b60 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:36:32.635082 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 496/991] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:36:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:36:35.085235 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2786f4-c5c0-4f95-9ee9-a8ca2a34bff0 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:36:35.085826 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2786f4-c5c0-4f95-9ee9-a8ca2a34bff0 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:36:35.085826 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2786f4-c5c0-4f95-9ee9-a8ca2a34bff0 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:36:35.087172 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 496/992] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:36:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:36:35.092333 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba0aaaf-e1bf-49ed-b998-365b6cbec908 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:36:35.092729 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba0aaaf-e1bf-49ed-b998-365b6cbec908 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:36:35.092889 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba0aaaf-e1bf-49ed-b998-365b6cbec908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:36:35.160737 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-2ba0aaaf-e1bf-49ed-b998-365b6cbec908 None None] MFA Rules not processed for user `d4cf2957a0f5441cb522c75512551dee`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:36:35.161975 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-2ba0aaaf-e1bf-49ed-b998-365b6cbec908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:36:35.213860 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:36:35.258550 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 497/993] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:36:35 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:36:35.541290 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2786f4-c5c0-4f95-9ee9-a8ca2a34bff0 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:36:35.542871 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-6a2786f4-c5c0-4f95-9ee9-a8ca2a34bff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:36:35.553106 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:36:35.645541 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2786f4-c5c0-4f95-9ee9-a8ca2a34bff0 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:36:35.717547 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347074c8-e6ff-4ec9-9eae-0b154dac0171 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:36:35.718106 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347074c8-e6ff-4ec9-9eae-0b154dac0171 service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:36:35.718384 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347074c8-e6ff-4ec9-9eae-0b154dac0171 service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:36:35.718654 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347074c8-e6ff-4ec9-9eae-0b154dac0171 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:36:35.720208 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-347074c8-e6ff-4ec9-9eae-0b154dac0171 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:36:35.780011 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:36:35.825796 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347074c8-e6ff-4ec9-9eae-0b154dac0171 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:36:35.827224 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347074c8-e6ff-4ec9-9eae-0b154dac0171 service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:36:35.844805 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 497/994] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:36:35 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:36:38.523446 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74564f1-1bd0-4061-9de4-49caeae97d75 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:36:38.523969 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74564f1-1bd0-4061-9de4-49caeae97d75 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:36:38.524089 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74564f1-1bd0-4061-9de4-49caeae97d75 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:36:38.525346 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 498/995] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:36:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 08:36:38.530669 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bf3618-9426-406f-9cbc-2e1e752473b4 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:36:38.531081 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bf3618-9426-406f-9cbc-2e1e752473b4 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:36:38.531221 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bf3618-9426-406f-9cbc-2e1e752473b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:36:38.595805 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-45bf3618-9426-406f-9cbc-2e1e752473b4 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:36:38.597004 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-45bf3618-9426-406f-9cbc-2e1e752473b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:36:38.690631 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 498/996] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:36:38 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:36:38.977217 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74564f1-1bd0-4061-9de4-49caeae97d75 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:36:38.978767 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-f74564f1-1bd0-4061-9de4-49caeae97d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:36:39.080631 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74564f1-1bd0-4061-9de4-49caeae97d75 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:36:39.151063 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c63b58-ed21-4c37-bcd5-38db65930eaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:36:39.151657 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c63b58-ed21-4c37-bcd5-38db65930eaf service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:36:39.151935 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c63b58-ed21-4c37-bcd5-38db65930eaf service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:36:39.152208 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c63b58-ed21-4c37-bcd5-38db65930eaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:36:39.153783 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-86c63b58-ed21-4c37-bcd5-38db65930eaf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:36:39.161269 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:36:39.260769 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c63b58-ed21-4c37-bcd5-38db65930eaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:36:39.262244 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c63b58-ed21-4c37-bcd5-38db65930eaf service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:36:39.280242 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 499/997] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:36:38 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:36:42.057526 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796d358d-4c76-4133-b944-63ec55a35446 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:36:42.058092 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796d358d-4c76-4133-b944-63ec55a35446 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:36:42.058203 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796d358d-4c76-4133-b944-63ec55a35446 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:36:42.059472 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 499/998] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:36:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:36:42.065716 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afec2e5-1239-49e8-874b-0c4ebce02b08 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:36:42.066343 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afec2e5-1239-49e8-874b-0c4ebce02b08 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:36:42.066772 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afec2e5-1239-49e8-874b-0c4ebce02b08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:36:42.142522 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-6afec2e5-1239-49e8-874b-0c4ebce02b08 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:36:42.143752 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-6afec2e5-1239-49e8-874b-0c4ebce02b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:36:42.206262 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:36:42.247084 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 500/999] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:36:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:36:42.544347 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6394be-f982-49e1-a2f5-6ad4495497f0 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:36:42.544636 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6394be-f982-49e1-a2f5-6ad4495497f0 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:36:42.544913 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6394be-f982-49e1-a2f5-6ad4495497f0 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:36:42.546611 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 500/1000] 10.222.0.30 () {54 vars in 964 bytes} [Sat Oct 15 08:36:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:36:42.553078 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3353e511-4af8-4fe2-a472-45cf43308f4a None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:36:42.553441 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3353e511-4af8-4fe2-a472-45cf43308f4a None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:36:42.553892 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3353e511-4af8-4fe2-a472-45cf43308f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:36:42.630657 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-3353e511-4af8-4fe2-a472-45cf43308f4a None None] MFA Rules not processed for user `8b11e349e4cb4558a4c75d843effc0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:36:42.631859 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-3353e511-4af8-4fe2-a472-45cf43308f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:36:42.705983 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 501/1001] 10.222.0.30 () {58 vars in 1061 bytes} [Sat Oct 15 08:36:42 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:36:42.715364 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6394be-f982-49e1-a2f5-6ad4495497f0 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:36:42.716985 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-5d6394be-f982-49e1-a2f5-6ad4495497f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:36:42.737080 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:36:42.822198 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6394be-f982-49e1-a2f5-6ad4495497f0 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:36:42.895699 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2c3804-2963-44ad-927a-eb05bef10660 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:36:42.896306 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c3804-2963-44ad-927a-eb05bef10660 service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:36:42.896838 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c3804-2963-44ad-927a-eb05bef10660 service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:36:42.897114 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c3804-2963-44ad-927a-eb05bef10660 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:36:42.898735 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-4a2c3804-2963-44ad-927a-eb05bef10660 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:36:42.970658 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:36:43.008756 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2c3804-2963-44ad-927a-eb05bef10660 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:36:43.010236 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2c3804-2963-44ad-927a-eb05bef10660 service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:36:43.029043 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 501/1002] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:36:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:36:45.464364 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6eb7ed-43dc-417a-b140-32422908c0d1 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:36:45.464857 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6eb7ed-43dc-417a-b140-32422908c0d1 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:36:45.464857 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6eb7ed-43dc-417a-b140-32422908c0d1 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:36:45.466302 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 502/1003] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:36:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:36:45.471469 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cde4ff-0db9-481f-a869-51d7efe94941 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:36:45.471899 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cde4ff-0db9-481f-a869-51d7efe94941 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:36:45.472033 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cde4ff-0db9-481f-a869-51d7efe94941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:36:45.540674 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-c9cde4ff-0db9-481f-a869-51d7efe94941 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:36:45.541913 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-c9cde4ff-0db9-481f-a869-51d7efe94941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:36:45.637571 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 502/1004] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:36:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:36:45.918445 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6eb7ed-43dc-417a-b140-32422908c0d1 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:36:45.919977 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-4c6eb7ed-43dc-417a-b140-32422908c0d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:36:45.981394 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:36:46.026502 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6eb7ed-43dc-417a-b140-32422908c0d1 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:36:46.100807 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9677e67-c987-4067-9d7b-7932e98b9628 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:36:46.101414 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9677e67-c987-4067-9d7b-7932e98b9628 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:36:46.101724 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9677e67-c987-4067-9d7b-7932e98b9628 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:36:46.101996 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9677e67-c987-4067-9d7b-7932e98b9628 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:36:46.103575 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-d9677e67-c987-4067-9d7b-7932e98b9628 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:36:46.213610 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:36:46.215919 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9677e67-c987-4067-9d7b-7932e98b9628 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:36:46.217429 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9677e67-c987-4067-9d7b-7932e98b9628 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:36:46.237591 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 503/1005] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:36:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:36:48.460193 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f9980-17c6-4c48-921b-f2822db15b33 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:36:48.460817 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f9980-17c6-4c48-921b-f2822db15b33 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:36:48.460817 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f9980-17c6-4c48-921b-f2822db15b33 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:36:48.462139 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 503/1006] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:36:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:36:48.467607 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1035329d-6e2a-4001-8e09-9c46d68962c3 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:36:48.468012 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1035329d-6e2a-4001-8e09-9c46d68962c3 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:36:48.468101 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1035329d-6e2a-4001-8e09-9c46d68962c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:36:48.538089 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-1035329d-6e2a-4001-8e09-9c46d68962c3 None None] MFA Rules not processed for user `d4cf2957a0f5441cb522c75512551dee`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:36:48.539359 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-1035329d-6e2a-4001-8e09-9c46d68962c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:36:48.640020 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 504/1007] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:36:48 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:36:48.920809 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11f9980-17c6-4c48-921b-f2822db15b33 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:36:48.922385 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-c11f9980-17c6-4c48-921b-f2822db15b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:36:48.952973 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:36:49.027907 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11f9980-17c6-4c48-921b-f2822db15b33 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:36:49.106914 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3714f27c-f5c3-49ab-b7ab-2ce376d84abc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:36:49.107529 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3714f27c-f5c3-49ab-b7ab-2ce376d84abc service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:36:49.107812 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3714f27c-f5c3-49ab-b7ab-2ce376d84abc service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:36:49.108088 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3714f27c-f5c3-49ab-b7ab-2ce376d84abc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:36:49.109698 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-3714f27c-f5c3-49ab-b7ab-2ce376d84abc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:36:49.190119 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:36:49.222680 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3714f27c-f5c3-49ab-b7ab-2ce376d84abc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:36:49.224137 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3714f27c-f5c3-49ab-b7ab-2ce376d84abc service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:36:49.242306 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 504/1008] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:36:48 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:36:51.634546 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8495ca36-f328-4fb8-a4dc-f8c760a3090c None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:36:51.635116 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8495ca36-f328-4fb8-a4dc-f8c760a3090c None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:36:51.635116 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8495ca36-f328-4fb8-a4dc-f8c760a3090c None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:36:51.636462 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 505/1009] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:36:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:36:51.641839 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979f48cc-6207-4c65-a19a-c74043d7accb None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:36:51.642250 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979f48cc-6207-4c65-a19a-c74043d7accb None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:36:51.642402 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979f48cc-6207-4c65-a19a-c74043d7accb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:36:51.707998 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-979f48cc-6207-4c65-a19a-c74043d7accb None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:36:51.709133 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-979f48cc-6207-4c65-a19a-c74043d7accb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:36:51.804665 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 505/1010] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:36:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:36:52.090717 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8495ca36-f328-4fb8-a4dc-f8c760a3090c None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:36:52.092280 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-8495ca36-f328-4fb8-a4dc-f8c760a3090c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:36:52.161935 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:36:52.195683 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8495ca36-f328-4fb8-a4dc-f8c760a3090c None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:36:52.266733 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0699d973-3f6c-4a51-a2c0-37d05a930ee3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:36:52.267332 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0699d973-3f6c-4a51-a2c0-37d05a930ee3 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:36:52.267613 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0699d973-3f6c-4a51-a2c0-37d05a930ee3 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:36:52.267890 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0699d973-3f6c-4a51-a2c0-37d05a930ee3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:36:52.270063 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-0699d973-3f6c-4a51-a2c0-37d05a930ee3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:36:52.375154 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0699d973-3f6c-4a51-a2c0-37d05a930ee3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:36:52.376505 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0699d973-3f6c-4a51-a2c0-37d05a930ee3 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:36:52.386846 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:36:52.396128 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 506/1011] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:36:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:36:54.573822 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed9cf13-f94c-4318-890c-dd9ba392770a None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:36:54.574369 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed9cf13-f94c-4318-890c-dd9ba392770a None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:36:54.574369 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed9cf13-f94c-4318-890c-dd9ba392770a None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:36:54.575729 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 506/1012] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:36:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:36:54.580994 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62333d22-eae6-4884-8d11-caf15cbc9289 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:36:54.581392 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62333d22-eae6-4884-8d11-caf15cbc9289 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:36:54.581577 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62333d22-eae6-4884-8d11-caf15cbc9289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:36:54.646746 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-62333d22-eae6-4884-8d11-caf15cbc9289 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:36:54.648195 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-62333d22-eae6-4884-8d11-caf15cbc9289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:36:54.743122 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 507/1013] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:36:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:36:55.027359 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed9cf13-f94c-4318-890c-dd9ba392770a None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:36:55.028887 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-2ed9cf13-f94c-4318-890c-dd9ba392770a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:36:55.068892 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:36:55.131612 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed9cf13-f94c-4318-890c-dd9ba392770a None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:36:55.203224 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5be244-1c1f-41fd-ae64-7587cfb10ec7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:36:55.203823 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5be244-1c1f-41fd-ae64-7587cfb10ec7 service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:36:55.204098 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5be244-1c1f-41fd-ae64-7587cfb10ec7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:36:55.204366 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5be244-1c1f-41fd-ae64-7587cfb10ec7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:36:55.205963 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-0a5be244-1c1f-41fd-ae64-7587cfb10ec7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:36:55.289134 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:36:55.314576 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5be244-1c1f-41fd-ae64-7587cfb10ec7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:36:55.316084 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5be244-1c1f-41fd-ae64-7587cfb10ec7 service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:36:55.333764 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 507/1014] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:36:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:36:57.774703 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03afae0-aeee-4fb4-a48b-e2979f9d0700 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:36:57.775306 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03afae0-aeee-4fb4-a48b-e2979f9d0700 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:36:57.775306 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03afae0-aeee-4fb4-a48b-e2979f9d0700 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:36:57.776604 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 508/1015] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:36:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:36:57.781643 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ff85e-0c6f-4519-994c-16cafc8e5a15 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:36:57.782095 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ff85e-0c6f-4519-994c-16cafc8e5a15 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:36:57.782203 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ff85e-0c6f-4519-994c-16cafc8e5a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:36:57.847325 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-515ff85e-0c6f-4519-994c-16cafc8e5a15 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:36:57.848521 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-515ff85e-0c6f-4519-994c-16cafc8e5a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:36:57.942815 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 508/1016] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:36:57 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:36:58.222417 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03afae0-aeee-4fb4-a48b-e2979f9d0700 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:36:58.223955 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-a03afae0-aeee-4fb4-a48b-e2979f9d0700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:36:58.298504 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:36:58.325737 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03afae0-aeee-4fb4-a48b-e2979f9d0700 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:36:58.399085 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a6c9fc-9aae-4133-afc1-95f75bb8f424 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:36:58.399839 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a6c9fc-9aae-4133-afc1-95f75bb8f424 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:36:58.399977 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a6c9fc-9aae-4133-afc1-95f75bb8f424 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:36:58.400255 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a6c9fc-9aae-4133-afc1-95f75bb8f424 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:36:58.401950 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-99a6c9fc-9aae-4133-afc1-95f75bb8f424 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:36:58.509787 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a6c9fc-9aae-4133-afc1-95f75bb8f424 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:36:58.511219 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a6c9fc-9aae-4133-afc1-95f75bb8f424 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:36:58.528901 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 509/1017] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:36:58 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:37:02.515353 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c212a-a300-4532-affe-ead022223fcc None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:02.515880 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c212a-a300-4532-affe-ead022223fcc None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:02.515880 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c212a-a300-4532-affe-ead022223fcc None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:02.517253 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 509/1018] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:37:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:37:02.523340 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af38c0f-4ea8-42c8-a05b-bcd76d3942a0 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:02.523772 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af38c0f-4ea8-42c8-a05b-bcd76d3942a0 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:02.523878 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af38c0f-4ea8-42c8-a05b-bcd76d3942a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:02.529261 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:37:02.590968 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-0af38c0f-4ea8-42c8-a05b-bcd76d3942a0 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:37:02.592058 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-0af38c0f-4ea8-42c8-a05b-bcd76d3942a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:37:02.688044 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 510/1019] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:37:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:37:02.970849 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89c212a-a300-4532-affe-ead022223fcc None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:37:02.972413 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-f89c212a-a300-4532-affe-ead022223fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:37:03.021521 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:37:03.078571 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89c212a-a300-4532-affe-ead022223fcc None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:37:03.152287 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9bed6b-1949-4a21-ae0f-867cd34f6f77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:37:03.152894 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9bed6b-1949-4a21-ae0f-867cd34f6f77 service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:03.153174 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9bed6b-1949-4a21-ae0f-867cd34f6f77 service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:03.153446 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9bed6b-1949-4a21-ae0f-867cd34f6f77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:03.155063 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-6d9bed6b-1949-4a21-ae0f-867cd34f6f77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:37:03.247120 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:37:03.263973 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9bed6b-1949-4a21-ae0f-867cd34f6f77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:37:03.265436 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9bed6b-1949-4a21-ae0f-867cd34f6f77 service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:37:03.285320 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 510/1020] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:37:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:37:06.075203 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c478bc0-69f5-4929-8294-0d9860ebbb79 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:06.075745 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c478bc0-69f5-4929-8294-0d9860ebbb79 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:06.075745 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c478bc0-69f5-4929-8294-0d9860ebbb79 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:06.077275 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 511/1021] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:37:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:37:06.083180 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767306d7-0be6-4f51-9a46-0108b7095a5c None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:06.083642 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767306d7-0be6-4f51-9a46-0108b7095a5c None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:06.083753 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767306d7-0be6-4f51-9a46-0108b7095a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:06.150224 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-767306d7-0be6-4f51-9a46-0108b7095a5c None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:37:06.151430 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-767306d7-0be6-4f51-9a46-0108b7095a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:37:06.246365 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 511/1022] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:37:06 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:37:06.531698 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c478bc0-69f5-4929-8294-0d9860ebbb79 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:37:06.533230 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-8c478bc0-69f5-4929-8294-0d9860ebbb79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:37:06.624292 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:37:06.644904 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c478bc0-69f5-4929-8294-0d9860ebbb79 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:37:06.720386 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7937ac-c4c2-4c78-b192-0083e0441a24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:37:06.720913 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7937ac-c4c2-4c78-b192-0083e0441a24 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:06.721193 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7937ac-c4c2-4c78-b192-0083e0441a24 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:06.721466 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7937ac-c4c2-4c78-b192-0083e0441a24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:06.723267 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-6d7937ac-c4c2-4c78-b192-0083e0441a24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:37:06.834150 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7937ac-c4c2-4c78-b192-0083e0441a24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:37:06.835585 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7937ac-c4c2-4c78-b192-0083e0441a24 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:37:06.854506 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 512/1023] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:37:06 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:37:10.935428 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce70e2c8-71f3-4e73-b465-1d377e167974 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:10.935949 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce70e2c8-71f3-4e73-b465-1d377e167974 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:10.936040 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce70e2c8-71f3-4e73-b465-1d377e167974 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:10.937349 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 512/1024] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:37:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:37:10.942892 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfaadeb-6510-4ce6-8008-0a4c86b4f59c None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:10.943379 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfaadeb-6510-4ce6-8008-0a4c86b4f59c None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:10.943562 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfaadeb-6510-4ce6-8008-0a4c86b4f59c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:10.959300 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:37:11.015111 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-2cfaadeb-6510-4ce6-8008-0a4c86b4f59c None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:37:11.016337 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-2cfaadeb-6510-4ce6-8008-0a4c86b4f59c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:37:11.115183 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 513/1025] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:37:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 08:37:11.401538 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce70e2c8-71f3-4e73-b465-1d377e167974 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:37:11.403177 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-ce70e2c8-71f3-4e73-b465-1d377e167974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:37:11.459929 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:37:11.505647 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce70e2c8-71f3-4e73-b465-1d377e167974 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:37:11.583951 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16531080-341a-43ec-b389-996ed05e9676 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:37:11.584572 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16531080-341a-43ec-b389-996ed05e9676 service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:11.584854 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16531080-341a-43ec-b389-996ed05e9676 service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:11.585130 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16531080-341a-43ec-b389-996ed05e9676 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:11.586865 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-16531080-341a-43ec-b389-996ed05e9676 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:37:11.694662 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:37:11.698769 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16531080-341a-43ec-b389-996ed05e9676 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:37:11.700216 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16531080-341a-43ec-b389-996ed05e9676 service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:37:11.717969 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 513/1026] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:37:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:37:16.608620 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46a2026-2605-49db-807b-8c7ba1e6b77b None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:16.609504 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46a2026-2605-49db-807b-8c7ba1e6b77b None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:16.609963 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46a2026-2605-49db-807b-8c7ba1e6b77b None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:16.611469 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 514/1027] 10.222.0.30 () {54 vars in 921 bytes} [Sat Oct 15 08:37:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:37:16.616884 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034952b7-e02b-4e5e-9003-504698867b32 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:16.617282 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034952b7-e02b-4e5e-9003-504698867b32 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:16.617585 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034952b7-e02b-4e5e-9003-504698867b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:16.768684 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-034952b7-e02b-4e5e-9003-504698867b32 None None] MFA Rules not processed for user `38986ca2612c4fc59d960cd693834f9b`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:37:16.770398 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-034952b7-e02b-4e5e-9003-504698867b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:37:16.890472 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 514/1028] 10.222.0.30 () {58 vars in 1018 bytes} [Sat Oct 15 08:37:16 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:37:20.858569 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a73a3c-e027-4200-a165-832298b37a7d None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:20.859174 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a73a3c-e027-4200-a165-832298b37a7d None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:20.859174 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a73a3c-e027-4200-a165-832298b37a7d None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:20.860450 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 515/1029] 10.222.0.30 () {54 vars in 921 bytes} [Sat Oct 15 08:37:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:37:20.865828 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc51a0-6b1a-49b9-8af1-4bea0fd4034d None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:20.866311 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc51a0-6b1a-49b9-8af1-4bea0fd4034d None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:20.866405 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc51a0-6b1a-49b9-8af1-4bea0fd4034d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:20.921629 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:37:20.947714 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-17fc51a0-6b1a-49b9-8af1-4bea0fd4034d None None] MFA Rules not processed for user `38986ca2612c4fc59d960cd693834f9b`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:37:20.949057 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-17fc51a0-6b1a-49b9-8af1-4bea0fd4034d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:37:21.061558 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 515/1030] 10.222.0.30 () {58 vars in 1018 bytes} [Sat Oct 15 08:37:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:37:21.113486 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=59 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:37:22.609375 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b896a0-2e8e-43fa-9da7-0a0f78ca5446 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:22.610055 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b896a0-2e8e-43fa-9da7-0a0f78ca5446 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:22.610055 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b896a0-2e8e-43fa-9da7-0a0f78ca5446 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:22.611599 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 516/1031] 10.222.0.30 () {54 vars in 921 bytes} [Sat Oct 15 08:37:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:37:22.617039 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3292fd23-4652-4c9a-855e-72a68f8e9ded None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:22.617489 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3292fd23-4652-4c9a-855e-72a68f8e9ded None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:22.617592 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3292fd23-4652-4c9a-855e-72a68f8e9ded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:22.696248 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-3292fd23-4652-4c9a-855e-72a68f8e9ded None None] MFA Rules not processed for user `38986ca2612c4fc59d960cd693834f9b`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:37:22.697342 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-3292fd23-4652-4c9a-855e-72a68f8e9ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:37:22.739942 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:37:22.804068 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 516/1032] 10.222.0.30 () {58 vars in 1018 bytes} [Sat Oct 15 08:37:22 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:37:26.378874 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa264d9c-b336-4fa0-bf8a-75dc93419d95 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:26.379417 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa264d9c-b336-4fa0-bf8a-75dc93419d95 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:26.379417 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa264d9c-b336-4fa0-bf8a-75dc93419d95 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:26.380666 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 517/1033] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:37:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:37:26.386344 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0a634-55c6-412b-bc1c-d713a693153d None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:26.387627 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0a634-55c6-412b-bc1c-d713a693153d None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:26.388166 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0a634-55c6-412b-bc1c-d713a693153d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:26.462819 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-3bd0a634-55c6-412b-bc1c-d713a693153d None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:37:26.463940 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-3bd0a634-55c6-412b-bc1c-d713a693153d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:37:26.566775 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 517/1034] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:37:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:37:26.868071 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cbf12c-2a98-4d5a-b8d5-158cfd067fb8 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:26.869268 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cbf12c-2a98-4d5a-b8d5-158cfd067fb8 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:26.869914 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cbf12c-2a98-4d5a-b8d5-158cfd067fb8 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:26.871763 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 518/1035] 10.222.0.30 () {54 vars in 961 bytes} [Sat Oct 15 08:37:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:37:26.882137 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f7863-2f0b-4237-8653-5c477ef900d7 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:26.883532 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f7863-2f0b-4237-8653-5c477ef900d7 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:26.884135 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f7863-2f0b-4237-8653-5c477ef900d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:26.904227 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:37:26.971777 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-038f7863-2f0b-4237-8653-5c477ef900d7 None None] MFA Rules not processed for user `af0a8af5cadc4cc1ab73e8eb2a20bb7e`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:37:26.973023 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-038f7863-2f0b-4237-8653-5c477ef900d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:37:27.096016 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 518/1036] 10.222.0.30 () {58 vars in 1058 bytes} [Sat Oct 15 08:37:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 08:37:27.103998 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cbf12c-2a98-4d5a-b8d5-158cfd067fb8 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:37:27.105517 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-75cbf12c-2a98-4d5a-b8d5-158cfd067fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:37:27.265084 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cbf12c-2a98-4d5a-b8d5-158cfd067fb8 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:37:27.346990 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:37:27.392942 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13841281-3e3a-4d4d-b4bf-58efe32491c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0a8af5cadc4cc1ab73e8eb2a20bb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:37:27.394028 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13841281-3e3a-4d4d-b4bf-58efe32491c0 service nova] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:27.394028 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13841281-3e3a-4d4d-b4bf-58efe32491c0 service nova] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:27.397398 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13841281-3e3a-4d4d-b4bf-58efe32491c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:27.397398 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-13841281-3e3a-4d4d-b4bf-58efe32491c0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:37:27.507268 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13841281-3e3a-4d4d-b4bf-58efe32491c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:37:27.509057 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13841281-3e3a-4d4d-b4bf-58efe32491c0 service nova] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:37:27.529958 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 519/1037] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:37:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:37:29.950253 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1e9ec4-e970-4240-b040-516816d088f7 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:29.950253 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1e9ec4-e970-4240-b040-516816d088f7 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:29.950253 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1e9ec4-e970-4240-b040-516816d088f7 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:29.951726 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 519/1038] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:37:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:37:29.958191 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803793db-e649-421a-a2d3-7b3d704421bc None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:29.958191 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803793db-e649-421a-a2d3-7b3d704421bc None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:29.959114 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803793db-e649-421a-a2d3-7b3d704421bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:30.035448 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-803793db-e649-421a-a2d3-7b3d704421bc None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:37:30.036986 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-803793db-e649-421a-a2d3-7b3d704421bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:37:30.079258 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:37:30.145758 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 520/1039] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:37:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:37:30.431226 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1e9ec4-e970-4240-b040-516816d088f7 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:37:30.433150 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-ee1e9ec4-e970-4240-b040-516816d088f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:37:30.513201 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:37:30.612578 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1e9ec4-e970-4240-b040-516816d088f7 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:37:30.736239 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec7da8e-49cd-451a-8af7-20b4106fef71 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0a8af5cadc4cc1ab73e8eb2a20bb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:37:30.736850 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec7da8e-49cd-451a-8af7-20b4106fef71 service nova] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:30.737129 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec7da8e-49cd-451a-8af7-20b4106fef71 service nova] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:30.738228 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec7da8e-49cd-451a-8af7-20b4106fef71 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:30.739842 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-dec7da8e-49cd-451a-8af7-20b4106fef71 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:37:30.762622 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:37:30.832002 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec7da8e-49cd-451a-8af7-20b4106fef71 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:37:30.833520 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec7da8e-49cd-451a-8af7-20b4106fef71 service nova] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:37:30.851641 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 520/1040] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:37:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:37:33.332971 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd1c4a7-b847-469e-862a-6d9ba8ac4b9d None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:33.333487 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd1c4a7-b847-469e-862a-6d9ba8ac4b9d None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:33.333487 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd1c4a7-b847-469e-862a-6d9ba8ac4b9d None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:33.334811 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 521/1041] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:37:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:37:33.340258 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee76f8-c544-4e74-94ab-de51b7ff3fdd None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:33.340700 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee76f8-c544-4e74-94ab-de51b7ff3fdd None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:33.340822 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee76f8-c544-4e74-94ab-de51b7ff3fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:33.409794 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-e2ee76f8-c544-4e74-94ab-de51b7ff3fdd None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:37:33.411006 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-e2ee76f8-c544-4e74-94ab-de51b7ff3fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:37:33.508898 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:37:33.521988 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 521/1042] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:37:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:37:33.803645 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa8c97f-6e17-4e5b-a12f-b01daea073a9 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:33.803941 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa8c97f-6e17-4e5b-a12f-b01daea073a9 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:33.804225 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa8c97f-6e17-4e5b-a12f-b01daea073a9 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:33.805813 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 522/1043] 10.222.0.30 () {54 vars in 961 bytes} [Sat Oct 15 08:37:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:37:33.811172 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96847286-9b3a-43c1-84d8-bbe8250fa522 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:33.811643 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96847286-9b3a-43c1-84d8-bbe8250fa522 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:33.812084 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96847286-9b3a-43c1-84d8-bbe8250fa522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:33.893176 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-96847286-9b3a-43c1-84d8-bbe8250fa522 None None] MFA Rules not processed for user `af0a8af5cadc4cc1ab73e8eb2a20bb7e`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:37:33.895384 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-96847286-9b3a-43c1-84d8-bbe8250fa522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:37:34.021171 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 522/1044] 10.222.0.30 () {58 vars in 1058 bytes} [Sat Oct 15 08:37:33 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:37:34.029517 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa8c97f-6e17-4e5b-a12f-b01daea073a9 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:37:34.031284 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-5fa8c97f-6e17-4e5b-a12f-b01daea073a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:37:34.193298 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa8c97f-6e17-4e5b-a12f-b01daea073a9 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:37:34.214568 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:37:34.314607 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0e6a9e-15ca-445d-83b6-ffe5fc800056 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0a8af5cadc4cc1ab73e8eb2a20bb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:37:34.315615 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e6a9e-15ca-445d-83b6-ffe5fc800056 service nova] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:34.315999 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e6a9e-15ca-445d-83b6-ffe5fc800056 service nova] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:34.316384 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e6a9e-15ca-445d-83b6-ffe5fc800056 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:34.318190 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-6e0e6a9e-15ca-445d-83b6-ffe5fc800056 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:37:34.424056 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0e6a9e-15ca-445d-83b6-ffe5fc800056 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:37:34.426510 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0e6a9e-15ca-445d-83b6-ffe5fc800056 service nova] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:37:34.457873 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 523/1045] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:37:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:37:34.536432 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebde4d8-1ac7-4b9b-ac7c-02f0881e498c None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:34.537023 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebde4d8-1ac7-4b9b-ac7c-02f0881e498c None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:34.537023 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebde4d8-1ac7-4b9b-ac7c-02f0881e498c None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:34.541711 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 523/1046] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:37:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:37:34.543921 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00242e83-68a3-4aa6-bfb5-d51cddae8360 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:34.544214 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00242e83-68a3-4aa6-bfb5-d51cddae8360 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:34.544540 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00242e83-68a3-4aa6-bfb5-d51cddae8360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:34.582517 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:37:34.624133 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-00242e83-68a3-4aa6-bfb5-d51cddae8360 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:37:34.625504 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-00242e83-68a3-4aa6-bfb5-d51cddae8360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:37:34.742544 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 524/1047] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:37:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:37:35.076032 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e7f093-5bcc-4e77-b3d5-8af9daa1e522 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:35.076598 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e7f093-5bcc-4e77-b3d5-8af9daa1e522 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:35.077080 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e7f093-5bcc-4e77-b3d5-8af9daa1e522 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:35.079254 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 524/1048] 10.222.0.30 () {54 vars in 963 bytes} [Sat Oct 15 08:37:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:37:35.084305 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a226db-2e89-4eb5-bf6f-844b0b8cc078 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:35.084589 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a226db-2e89-4eb5-bf6f-844b0b8cc078 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:35.084860 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a226db-2e89-4eb5-bf6f-844b0b8cc078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:35.165226 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-93a226db-2e89-4eb5-bf6f-844b0b8cc078 None None] MFA Rules not processed for user `a649c5fedad847fb9e784e779da3a24e`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:37:35.166485 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-93a226db-2e89-4eb5-bf6f-844b0b8cc078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:37:35.216449 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:37:35.248349 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 525/1049] 10.222.0.30 () {58 vars in 1060 bytes} [Sat Oct 15 08:37:35 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:37:35.255995 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e7f093-5bcc-4e77-b3d5-8af9daa1e522 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:37:35.257643 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-24e7f093-5bcc-4e77-b3d5-8af9daa1e522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:37:35.312472 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:37:35.372513 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e7f093-5bcc-4e77-b3d5-8af9daa1e522 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:37:35.461486 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c3d648-2d07-40e1-81e4-40c127e15ad6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:37:35.462148 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3d648-2d07-40e1-81e4-40c127e15ad6 service cinder] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:35.462434 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3d648-2d07-40e1-81e4-40c127e15ad6 service cinder] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:35.462712 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3d648-2d07-40e1-81e4-40c127e15ad6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:35.464335 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-11c3d648-2d07-40e1-81e4-40c127e15ad6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:37:35.582063 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:37:35.596506 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c3d648-2d07-40e1-81e4-40c127e15ad6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:37:35.598651 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c3d648-2d07-40e1-81e4-40c127e15ad6 service cinder] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:37:35.616889 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 525/1050] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:37:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 08:37:36.705708 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c8465-06ae-4b62-8460-f1230c6bb023 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:36.706779 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c8465-06ae-4b62-8460-f1230c6bb023 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:36.706779 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c8465-06ae-4b62-8460-f1230c6bb023 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:36.708079 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 526/1051] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:37:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:37:36.713456 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318f6fda-1a8a-4fd9-90aa-21807ce77332 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:36.713892 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318f6fda-1a8a-4fd9-90aa-21807ce77332 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:36.714057 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318f6fda-1a8a-4fd9-90aa-21807ce77332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:36.780292 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-318f6fda-1a8a-4fd9-90aa-21807ce77332 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:37:36.781492 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-318f6fda-1a8a-4fd9-90aa-21807ce77332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:37:36.877811 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 526/1052] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:37:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:37:37.170030 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689c8465-06ae-4b62-8460-f1230c6bb023 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:37:37.171581 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-689c8465-06ae-4b62-8460-f1230c6bb023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:37:37.323994 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689c8465-06ae-4b62-8460-f1230c6bb023 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:37:37.376079 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:37:37.442076 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d6a4a2-4f92-4826-9e5b-78dd90ea8c69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0a8af5cadc4cc1ab73e8eb2a20bb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:37:37.442754 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d6a4a2-4f92-4826-9e5b-78dd90ea8c69 service nova] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:37.443036 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d6a4a2-4f92-4826-9e5b-78dd90ea8c69 service nova] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:37.443305 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d6a4a2-4f92-4826-9e5b-78dd90ea8c69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:37.444894 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-73d6a4a2-4f92-4826-9e5b-78dd90ea8c69 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:37:37.542901 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d6a4a2-4f92-4826-9e5b-78dd90ea8c69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:37:37.544607 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d6a4a2-4f92-4826-9e5b-78dd90ea8c69 service nova] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:37:37.564027 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 527/1053] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:37:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:37:39.793087 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b1e66-7cb1-41af-8f90-8b056c27f093 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:39.793655 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b1e66-7cb1-41af-8f90-8b056c27f093 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:39.793989 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b1e66-7cb1-41af-8f90-8b056c27f093 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:39.795505 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 527/1054] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:37:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:37:39.801377 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ded90d-35f2-46b5-af00-73af21463a39 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:39.801839 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ded90d-35f2-46b5-af00-73af21463a39 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:39.802017 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ded90d-35f2-46b5-af00-73af21463a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:39.871129 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-26ded90d-35f2-46b5-af00-73af21463a39 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:37:39.872707 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-26ded90d-35f2-46b5-af00-73af21463a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:37:39.882385 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:37:39.981231 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 528/1055] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:37:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:37:40.274551 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07b1e66-7cb1-41af-8f90-8b056c27f093 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:37:40.276067 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-b07b1e66-7cb1-41af-8f90-8b056c27f093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:37:40.339139 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:37:40.435136 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07b1e66-7cb1-41af-8f90-8b056c27f093 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:37:40.556436 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad95da03-9d95-4277-8981-2a1bf4f5a9f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0a8af5cadc4cc1ab73e8eb2a20bb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:37:40.557043 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad95da03-9d95-4277-8981-2a1bf4f5a9f2 service nova] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:40.557383 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad95da03-9d95-4277-8981-2a1bf4f5a9f2 service nova] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:40.557667 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad95da03-9d95-4277-8981-2a1bf4f5a9f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:40.559276 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-ad95da03-9d95-4277-8981-2a1bf4f5a9f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:37:40.579108 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:37:40.655809 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad95da03-9d95-4277-8981-2a1bf4f5a9f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:37:40.657420 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad95da03-9d95-4277-8981-2a1bf4f5a9f2 service nova] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:37:40.679281 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 528/1056] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:37:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:37:42.888132 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab63080-3ad0-4915-bf46-38e4998e756f None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:42.888611 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab63080-3ad0-4915-bf46-38e4998e756f None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:42.888694 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab63080-3ad0-4915-bf46-38e4998e756f None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:42.894015 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 529/1057] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:37:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:37:42.895553 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390fc1a1-c1b9-4ec0-90e5-0a9510b6422c None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:42.895981 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390fc1a1-c1b9-4ec0-90e5-0a9510b6422c None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:42.896155 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390fc1a1-c1b9-4ec0-90e5-0a9510b6422c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:42.971718 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-390fc1a1-c1b9-4ec0-90e5-0a9510b6422c None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:37:42.972895 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-390fc1a1-c1b9-4ec0-90e5-0a9510b6422c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:37:43.083325 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:37:43.090010 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 529/1058] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:37:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 08:37:43.390589 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab63080-3ad0-4915-bf46-38e4998e756f None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:37:43.392739 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-dab63080-3ad0-4915-bf46-38e4998e756f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:37:43.550878 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:37:43.564814 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab63080-3ad0-4915-bf46-38e4998e756f None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:37:43.691747 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfde1e9-53b0-4257-9e05-613baa97449c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0a8af5cadc4cc1ab73e8eb2a20bb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:37:43.692314 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfde1e9-53b0-4257-9e05-613baa97449c service nova] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:43.692614 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfde1e9-53b0-4257-9e05-613baa97449c service nova] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:43.693574 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfde1e9-53b0-4257-9e05-613baa97449c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:43.695299 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-0bfde1e9-53b0-4257-9e05-613baa97449c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:37:43.792379 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfde1e9-53b0-4257-9e05-613baa97449c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:37:43.794102 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfde1e9-53b0-4257-9e05-613baa97449c service nova] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:37:43.804593 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:37:43.813591 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 530/1059] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:37:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:37:46.101550 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fa269f-3a9f-4830-b854-0f3f0517397d None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:46.102112 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fa269f-3a9f-4830-b854-0f3f0517397d None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:46.102112 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fa269f-3a9f-4830-b854-0f3f0517397d None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:46.103485 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 530/1060] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:37:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:37:46.109154 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ed4c7-78ea-494b-a399-6aeb9826cdd5 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:46.109591 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ed4c7-78ea-494b-a399-6aeb9826cdd5 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:46.109770 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ed4c7-78ea-494b-a399-6aeb9826cdd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:46.181301 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-071ed4c7-78ea-494b-a399-6aeb9826cdd5 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:37:46.182994 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-071ed4c7-78ea-494b-a399-6aeb9826cdd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:37:46.303363 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 531/1061] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:37:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:37:46.597610 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fa269f-3a9f-4830-b854-0f3f0517397d None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:37:46.599446 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-86fa269f-3a9f-4830-b854-0f3f0517397d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:37:46.769378 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fa269f-3a9f-4830-b854-0f3f0517397d None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:37:46.838839 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:37:46.887296 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c182fc3f-3294-4c57-b741-7383cb062ed9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0a8af5cadc4cc1ab73e8eb2a20bb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:37:46.887906 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c182fc3f-3294-4c57-b741-7383cb062ed9 service nova] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:46.888194 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c182fc3f-3294-4c57-b741-7383cb062ed9 service nova] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:46.888503 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c182fc3f-3294-4c57-b741-7383cb062ed9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:46.890128 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-c182fc3f-3294-4c57-b741-7383cb062ed9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:37:46.984043 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c182fc3f-3294-4c57-b741-7383cb062ed9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:37:46.986453 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c182fc3f-3294-4c57-b741-7383cb062ed9 service nova] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:37:47.012115 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 531/1062] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:37:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:37:49.247754 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9371af-cb7c-49f2-83b2-5db2d89a465b None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:49.248322 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9371af-cb7c-49f2-83b2-5db2d89a465b None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:49.248322 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9371af-cb7c-49f2-83b2-5db2d89a465b None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:49.249672 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 532/1063] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:37:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:37:49.256195 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc0d5b-2190-4881-bff6-4bcf957a43c1 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:49.256619 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc0d5b-2190-4881-bff6-4bcf957a43c1 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:49.256732 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc0d5b-2190-4881-bff6-4bcf957a43c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:49.325009 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-59cc0d5b-2190-4881-bff6-4bcf957a43c1 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:37:49.326326 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-59cc0d5b-2190-4881-bff6-4bcf957a43c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:37:49.360605 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:37:49.437611 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 532/1064] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:37:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 08:37:49.735529 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9371af-cb7c-49f2-83b2-5db2d89a465b None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:37:49.737627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-fd9371af-cb7c-49f2-83b2-5db2d89a465b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:37:49.816314 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:37:49.900572 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9371af-cb7c-49f2-83b2-5db2d89a465b None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:37:50.025491 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037f98b9-2358-42ae-b62c-edf66320a4d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0a8af5cadc4cc1ab73e8eb2a20bb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:37:50.026484 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f98b9-2358-42ae-b62c-edf66320a4d0 service nova] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:50.026896 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f98b9-2358-42ae-b62c-edf66320a4d0 service nova] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:50.027325 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f98b9-2358-42ae-b62c-edf66320a4d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:50.029599 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-037f98b9-2358-42ae-b62c-edf66320a4d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:37:50.072889 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:37:50.129885 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037f98b9-2358-42ae-b62c-edf66320a4d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:37:50.131595 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037f98b9-2358-42ae-b62c-edf66320a4d0 service nova] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:37:50.151635 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 533/1065] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:37:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:37:52.178531 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c2a699-c026-4901-9852-af031ac14372 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:52.179116 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c2a699-c026-4901-9852-af031ac14372 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:52.179116 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c2a699-c026-4901-9852-af031ac14372 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:52.180457 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 533/1066] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:37:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:37:52.185789 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7edcf7-ea05-412d-a632-dc6b86f7fa74 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:52.186274 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7edcf7-ea05-412d-a632-dc6b86f7fa74 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:52.186382 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7edcf7-ea05-412d-a632-dc6b86f7fa74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:52.260188 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-ba7edcf7-ea05-412d-a632-dc6b86f7fa74 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:37:52.261452 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-ba7edcf7-ea05-412d-a632-dc6b86f7fa74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:37:52.362914 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba6dbee-1ff4-45cd-bb4e-acd9435d3d7a None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:52.363193 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba6dbee-1ff4-45cd-bb4e-acd9435d3d7a None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:52.363468 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba6dbee-1ff4-45cd-bb4e-acd9435d3d7a None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:52.364832 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 534/1067] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:37:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:37:52.370037 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765e7a23-8aab-4491-9ba6-043bb99bc6af None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:52.370329 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765e7a23-8aab-4491-9ba6-043bb99bc6af None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:52.370606 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765e7a23-8aab-4491-9ba6-043bb99bc6af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:52.401727 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 534/1068] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:37:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 08:37:52.465206 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-765e7a23-8aab-4491-9ba6-043bb99bc6af None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:37:52.466674 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-765e7a23-8aab-4491-9ba6-043bb99bc6af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:37:52.586470 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 535/1069] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:37:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 08:37:52.695053 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e47e485-6a4e-4348-85ff-1fba3520b6f7 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:52.695368 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e47e485-6a4e-4348-85ff-1fba3520b6f7 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:52.695689 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e47e485-6a4e-4348-85ff-1fba3520b6f7 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:52.697186 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 535/1070] 10.222.0.30 () {54 vars in 962 bytes} [Sat Oct 15 08:37:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:37:52.702764 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f33eb5-bc81-4736-a17b-3bb6b1d47d75 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:52.703077 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f33eb5-bc81-4736-a17b-3bb6b1d47d75 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:52.703398 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f33eb5-bc81-4736-a17b-3bb6b1d47d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:52.721139 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:37:52.786518 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-66f33eb5-bc81-4736-a17b-3bb6b1d47d75 None None] MFA Rules not processed for user `71692d05fd324679b1598feb52b3fe6d`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:37:52.787748 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-66f33eb5-bc81-4736-a17b-3bb6b1d47d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:37:52.864418 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 536/1071] 10.222.0.30 () {58 vars in 1059 bytes} [Sat Oct 15 08:37:52 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 08:37:52.873617 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e47e485-6a4e-4348-85ff-1fba3520b6f7 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:37:52.875408 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-1e47e485-6a4e-4348-85ff-1fba3520b6f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:37:52.891210 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:37:52.902486 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f33eb5-bc81-4736-a17b-3bb6b1d47d75 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:37:52.905195 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-66f33eb5-bc81-4736-a17b-3bb6b1d47d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:37:52.993176 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e47e485-6a4e-4348-85ff-1fba3520b6f7 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:37:53.015005 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:37:53.030386 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f33eb5-bc81-4736-a17b-3bb6b1d47d75 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:37:53.072819 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473652e3-bb1f-423a-84d0-5245d18b19c7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71692d05fd324679b1598feb52b3fe6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:37:53.073475 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473652e3-bb1f-423a-84d0-5245d18b19c7 service glance] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:53.073807 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473652e3-bb1f-423a-84d0-5245d18b19c7 service glance] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:53.074110 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473652e3-bb1f-423a-84d0-5245d18b19c7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:53.075834 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-473652e3-bb1f-423a-84d0-5245d18b19c7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:37:53.147804 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:37:53.152880 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25627f6-3690-4adc-b191-4b889140e91c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0a8af5cadc4cc1ab73e8eb2a20bb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:37:53.153532 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25627f6-3690-4adc-b191-4b889140e91c service nova] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:53.153857 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25627f6-3690-4adc-b191-4b889140e91c service nova] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:53.154152 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25627f6-3690-4adc-b191-4b889140e91c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:53.155893 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-e25627f6-3690-4adc-b191-4b889140e91c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:37:53.193838 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473652e3-bb1f-423a-84d0-5245d18b19c7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:37:53.195301 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473652e3-bb1f-423a-84d0-5245d18b19c7 service glance] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:37:53.214215 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 536/1072] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:37:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:37:53.251182 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25627f6-3690-4adc-b191-4b889140e91c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:37:53.253389 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25627f6-3690-4adc-b191-4b889140e91c service nova] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:37:53.264460 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:37:53.272983 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 537/1073] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:37:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:37:53.980016 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b752dc0b-b9a1-486e-8703-086b4542ae62 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:53.980422 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b752dc0b-b9a1-486e-8703-086b4542ae62 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:53.980786 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b752dc0b-b9a1-486e-8703-086b4542ae62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:54.052538 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-b752dc0b-b9a1-486e-8703-086b4542ae62 None None] MFA Rules not processed for user `c5b2a0e623c348ad94d4820080677be6`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:37:54.053906 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-b752dc0b-b9a1-486e-8703-086b4542ae62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:37:54.142084 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 537/1074] 10.222.0.30 () {58 vars in 1009 bytes} [Sat Oct 15 08:37:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:37:54.190595 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51398276-6a5c-4cd9-ac35-001acc54a797 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:54.190887 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51398276-6a5c-4cd9-ac35-001acc54a797 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:54.191196 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51398276-6a5c-4cd9-ac35-001acc54a797 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:54.192609 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 538/1075] 10.222.0.30 () {54 vars in 943 bytes} [Sat Oct 15 08:37:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:37:54.199152 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5373fd-20e5-4de1-8281-45b122c1e712 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:54.199626 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5373fd-20e5-4de1-8281-45b122c1e712 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:54.199725 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5373fd-20e5-4de1-8281-45b122c1e712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:54.246917 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:37:54.282699 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-1b5373fd-20e5-4de1-8281-45b122c1e712 None None] MFA Rules not processed for user `6e3009e20e4c49e5a74f7fd793b2c329`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:37:54.283918 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-1b5373fd-20e5-4de1-8281-45b122c1e712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:37:54.396235 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 538/1076] 10.222.0.30 () {58 vars in 1040 bytes} [Sat Oct 15 08:37:54 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:37:54.405937 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51398276-6a5c-4cd9-ac35-001acc54a797 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:37:54.407540 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-51398276-6a5c-4cd9-ac35-001acc54a797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:37:54.542724 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51398276-6a5c-4cd9-ac35-001acc54a797 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:37:54.622162 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:37:54.646002 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f86475-b90a-483e-b899-5369a3680899 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e3009e20e4c49e5a74f7fd793b2c329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:37:54.646612 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f86475-b90a-483e-b899-5369a3680899 service swift] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:54.646891 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f86475-b90a-483e-b899-5369a3680899 service swift] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:54.647164 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f86475-b90a-483e-b899-5369a3680899 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:54.649047 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-77f86475-b90a-483e-b899-5369a3680899 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:37:54.732917 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f86475-b90a-483e-b899-5369a3680899 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:37:54.734464 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f86475-b90a-483e-b899-5369a3680899 service swift] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:37:54.742827 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 539/1077] 10.222.0.30 () {60 vars in 1384 bytes} [Sat Oct 15 08:37:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 08:37:55.514840 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4318195c-a581-40c7-b54f-9ff4a1407866 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:55.515427 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4318195c-a581-40c7-b54f-9ff4a1407866 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:55.515427 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4318195c-a581-40c7-b54f-9ff4a1407866 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:55.516725 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 539/1078] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:37:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:37:55.522506 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b73f1bb-7b22-4095-9b1a-7a4100e5727f None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:55.522934 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b73f1bb-7b22-4095-9b1a-7a4100e5727f None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:55.523086 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b73f1bb-7b22-4095-9b1a-7a4100e5727f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:55.594744 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-0b73f1bb-7b22-4095-9b1a-7a4100e5727f None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:37:55.595964 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-0b73f1bb-7b22-4095-9b1a-7a4100e5727f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:37:55.669187 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:37:55.705524 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 540/1079] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:37:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:37:55.994829 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4318195c-a581-40c7-b54f-9ff4a1407866 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:37:55.996464 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-4318195c-a581-40c7-b54f-9ff4a1407866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:37:56.091718 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:37:56.156376 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4318195c-a581-40c7-b54f-9ff4a1407866 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:37:56.289621 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b224af-aec7-4f1a-88c8-b11c3c06dc8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0a8af5cadc4cc1ab73e8eb2a20bb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:37:56.290258 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b224af-aec7-4f1a-88c8-b11c3c06dc8d service nova] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:56.290535 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b224af-aec7-4f1a-88c8-b11c3c06dc8d service nova] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:56.290809 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b224af-aec7-4f1a-88c8-b11c3c06dc8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:56.292385 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-61b224af-aec7-4f1a-88c8-b11c3c06dc8d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:37:56.355530 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:37:56.422300 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b224af-aec7-4f1a-88c8-b11c3c06dc8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:37:56.423406 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b224af-aec7-4f1a-88c8-b11c3c06dc8d service nova] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:37:56.453640 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 540/1080] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:37:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:37:58.691188 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e925e760-a583-4ad3-ad9f-190b78394914 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:58.691704 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e925e760-a583-4ad3-ad9f-190b78394914 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:58.691704 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e925e760-a583-4ad3-ad9f-190b78394914 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:58.693003 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 541/1081] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:37:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:37:58.698217 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b89106c-2a04-42ca-ac22-0270f26a59e0 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:58.698679 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b89106c-2a04-42ca-ac22-0270f26a59e0 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:58.698771 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b89106c-2a04-42ca-ac22-0270f26a59e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:58.772724 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-5b89106c-2a04-42ca-ac22-0270f26a59e0 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:37:58.773081 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-5b89106c-2a04-42ca-ac22-0270f26a59e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:37:58.885757 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 541/1082] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:37:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:37:59.191368 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e925e760-a583-4ad3-ad9f-190b78394914 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:37:59.192881 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-e925e760-a583-4ad3-ad9f-190b78394914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:37:59.350815 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e925e760-a583-4ad3-ad9f-190b78394914 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:37:59.413427 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:37:59.482434 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ce152b-a2d8-4738-a33b-557d94e54191 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0a8af5cadc4cc1ab73e8eb2a20bb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:37:59.483130 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ce152b-a2d8-4738-a33b-557d94e54191 service nova] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:37:59.483414 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ce152b-a2d8-4738-a33b-557d94e54191 service nova] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:37:59.483688 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ce152b-a2d8-4738-a33b-557d94e54191 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:37:59.485306 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-93ce152b-a2d8-4738-a33b-557d94e54191 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:37:59.588728 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ce152b-a2d8-4738-a33b-557d94e54191 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:37:59.590366 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ce152b-a2d8-4738-a33b-557d94e54191 service nova] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:37:59.608760 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 542/1083] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:37:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:38:01.808685 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f189438-791b-41e6-8233-549e79c4b4aa None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:38:01.809278 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f189438-791b-41e6-8233-549e79c4b4aa None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:38:01.809278 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f189438-791b-41e6-8233-549e79c4b4aa None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:38:01.811504 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 542/1084] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:38:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:38:01.816082 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b903b3d1-807f-4070-9361-9d2abc732949 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:38:01.816546 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b903b3d1-807f-4070-9361-9d2abc732949 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:38:01.816768 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b903b3d1-807f-4070-9361-9d2abc732949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:38:01.886171 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-b903b3d1-807f-4070-9361-9d2abc732949 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:38:01.887713 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-b903b3d1-807f-4070-9361-9d2abc732949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:38:01.896829 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:38:02.001831 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 543/1085] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:38:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:38:02.296636 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f189438-791b-41e6-8233-549e79c4b4aa None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:38:02.298236 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-8f189438-791b-41e6-8233-549e79c4b4aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:38:02.343468 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:38:02.470269 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f189438-791b-41e6-8233-549e79c4b4aa None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:38:02.586489 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:38:02.595233 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a51cfc2-03f4-433c-9d4c-488b70fef344 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0a8af5cadc4cc1ab73e8eb2a20bb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:38:02.596043 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a51cfc2-03f4-433c-9d4c-488b70fef344 service nova] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:38:02.596328 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a51cfc2-03f4-433c-9d4c-488b70fef344 service nova] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:38:02.597144 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a51cfc2-03f4-433c-9d4c-488b70fef344 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:38:02.598828 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-1a51cfc2-03f4-433c-9d4c-488b70fef344 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:38:02.693129 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a51cfc2-03f4-433c-9d4c-488b70fef344 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:38:02.694816 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a51cfc2-03f4-433c-9d4c-488b70fef344 service nova] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:38:02.718640 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 543/1086] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:38:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:38:11.998026 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=39 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:38:12.710451 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=48 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:38:17.323006 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc77bea0-6dd4-4d10-a8c8-3adb3c7f3b8c None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:38:17.323569 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc77bea0-6dd4-4d10-a8c8-3adb3c7f3b8c None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:38:17.323569 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc77bea0-6dd4-4d10-a8c8-3adb3c7f3b8c None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:38:17.324912 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 544/1087] 10.222.0.30 () {54 vars in 918 bytes} [Sat Oct 15 08:38:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:38:17.330096 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520900da-67a2-4b70-a1bd-06563d545d43 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:38:17.330539 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520900da-67a2-4b70-a1bd-06563d545d43 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:38:17.330666 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520900da-67a2-4b70-a1bd-06563d545d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:38:17.405543 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-520900da-67a2-4b70-a1bd-06563d545d43 None None] MFA Rules not processed for user `38986ca2612c4fc59d960cd693834f9b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:38:17.406785 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-520900da-67a2-4b70-a1bd-06563d545d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:38:17.511107 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 544/1088] 10.222.0.30 () {58 vars in 1015 bytes} [Sat Oct 15 08:38:17 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:38:17.525622 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a0119-def7-4a3f-9d4a-20c77ac28cea None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:38:17.525944 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a0119-def7-4a3f-9d4a-20c77ac28cea None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:38:17.526219 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a0119-def7-4a3f-9d4a-20c77ac28cea None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:38:17.527546 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 545/1089] 10.222.0.30 () {54 vars in 918 bytes} [Sat Oct 15 08:38:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:38:17.532046 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccb0f6f-2dda-4cb8-b86c-f3303d3c09f5 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:38:17.532337 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccb0f6f-2dda-4cb8-b86c-f3303d3c09f5 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:38:17.532647 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccb0f6f-2dda-4cb8-b86c-f3303d3c09f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:38:17.605115 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:38:17.607729 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-8ccb0f6f-2dda-4cb8-b86c-f3303d3c09f5 None None] MFA Rules not processed for user `af0a8af5cadc4cc1ab73e8eb2a20bb7e`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:38:17.609478 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-8ccb0f6f-2dda-4cb8-b86c-f3303d3c09f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:38:17.723601 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 545/1090] 10.222.0.30 () {58 vars in 1015 bytes} [Sat Oct 15 08:38:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:38:20.277467 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c390918-6935-4336-828b-b68f282adcf5 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:38:20.278035 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c390918-6935-4336-828b-b68f282adcf5 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:38:20.278035 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c390918-6935-4336-828b-b68f282adcf5 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:38:20.279391 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 546/1091] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 15 08:38:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:38:20.284872 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f503582c-b26a-4964-a3dc-21fb78cf9728 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:38:20.285306 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f503582c-b26a-4964-a3dc-21fb78cf9728 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:38:20.285444 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f503582c-b26a-4964-a3dc-21fb78cf9728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:38:20.351372 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-f503582c-b26a-4964-a3dc-21fb78cf9728 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:38:20.352577 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-f503582c-b26a-4964-a3dc-21fb78cf9728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:38:20.410532 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:38:20.453042 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 546/1092] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 15 08:38:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 08:38:20.740684 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c390918-6935-4336-828b-b68f282adcf5 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:38:20.742672 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-9c390918-6935-4336-828b-b68f282adcf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:38:20.886371 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c390918-6935-4336-828b-b68f282adcf5 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:38:20.959084 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:38:20.998911 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46224660-bb84-4b4f-80e3-f146ac40b890 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0a8af5cadc4cc1ab73e8eb2a20bb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:38:20.999516 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46224660-bb84-4b4f-80e3-f146ac40b890 service nova] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:38:20.999792 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46224660-bb84-4b4f-80e3-f146ac40b890 service nova] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:38:21.000067 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46224660-bb84-4b4f-80e3-f146ac40b890 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:38:21.001631 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-46224660-bb84-4b4f-80e3-f146ac40b890 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:38:21.091041 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46224660-bb84-4b4f-80e3-f146ac40b890 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:38:21.092637 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46224660-bb84-4b4f-80e3-f146ac40b890 service nova] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:38:21.110671 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 547/1093] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:38:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:38:22.237994 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d693a953-7cdf-413e-b4da-5bcdcd8fc944 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:38:22.238593 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d693a953-7cdf-413e-b4da-5bcdcd8fc944 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:38:22.238593 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d693a953-7cdf-413e-b4da-5bcdcd8fc944 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:38:22.239898 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 547/1094] 127.0.0.1 () {54 vars in 901 bytes} [Sat Oct 15 08:38:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:38:22.246042 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d0fe8-e472-4d2d-949d-1957e7110892 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:38:22.246460 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d0fe8-e472-4d2d-949d-1957e7110892 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:38:22.246609 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d0fe8-e472-4d2d-949d-1957e7110892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:38:22.321617 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-431d0fe8-e472-4d2d-949d-1957e7110892 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:38:22.322851 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-431d0fe8-e472-4d2d-949d-1957e7110892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:38:22.351225 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:38:22.428922 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 548/1095] 127.0.0.1 () {58 vars in 998 bytes} [Sat Oct 15 08:38:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:38:22.578345 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d693a953-7cdf-413e-b4da-5bcdcd8fc944 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:38:22.579870 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-d693a953-7cdf-413e-b4da-5bcdcd8fc944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:38:22.729092 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d693a953-7cdf-413e-b4da-5bcdcd8fc944 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:38:22.767836 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:38:22.841342 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca2f056-3645-4c31-acfe-24dea0fb3c4d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0a8af5cadc4cc1ab73e8eb2a20bb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:38:22.841905 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2f056-3645-4c31-acfe-24dea0fb3c4d service nova] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:38:22.842187 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2f056-3645-4c31-acfe-24dea0fb3c4d service nova] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:38:22.842461 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2f056-3645-4c31-acfe-24dea0fb3c4d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:38:22.844038 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-fca2f056-3645-4c31-acfe-24dea0fb3c4d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:38:22.931400 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca2f056-3645-4c31-acfe-24dea0fb3c4d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:38:22.932965 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca2f056-3645-4c31-acfe-24dea0fb3c4d service nova] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:38:22.950683 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 548/1096] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:38:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:38:24.073746 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76376e62-a094-47eb-9f6b-f7720de00be7 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:38:24.074353 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76376e62-a094-47eb-9f6b-f7720de00be7 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:38:24.074450 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76376e62-a094-47eb-9f6b-f7720de00be7 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:38:24.075940 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 549/1097] 127.0.0.1 () {54 vars in 901 bytes} [Sat Oct 15 08:38:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:38:24.083019 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa931de-656e-4d3a-a5c9-8ced4861be10 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:38:24.083441 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa931de-656e-4d3a-a5c9-8ced4861be10 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:38:24.083585 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa931de-656e-4d3a-a5c9-8ced4861be10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:38:24.137034 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:38:24.160527 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-4aa931de-656e-4d3a-a5c9-8ced4861be10 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:38:24.161759 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-4aa931de-656e-4d3a-a5c9-8ced4861be10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:38:24.256037 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 549/1098] 127.0.0.1 () {58 vars in 998 bytes} [Sat Oct 15 08:38:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:38:24.409473 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76376e62-a094-47eb-9f6b-f7720de00be7 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:38:24.411024 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-76376e62-a094-47eb-9f6b-f7720de00be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:38:24.554603 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76376e62-a094-47eb-9f6b-f7720de00be7 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:38:24.566363 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:38:24.668306 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f0c8dd-d60e-4605-bcb9-c347c660e802 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0a8af5cadc4cc1ab73e8eb2a20bb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:38:24.668822 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f0c8dd-d60e-4605-bcb9-c347c660e802 service nova] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:38:24.669099 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f0c8dd-d60e-4605-bcb9-c347c660e802 service nova] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:38:24.669367 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f0c8dd-d60e-4605-bcb9-c347c660e802 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:38:24.670970 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-b0f0c8dd-d60e-4605-bcb9-c347c660e802 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:38:24.759527 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f0c8dd-d60e-4605-bcb9-c347c660e802 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:38:24.761108 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f0c8dd-d60e-4605-bcb9-c347c660e802 service nova] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:38:24.779226 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 550/1099] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:38:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:38:25.880085 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec858d-8829-4bfd-9271-129627aae0d7 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:38:25.880627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec858d-8829-4bfd-9271-129627aae0d7 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:38:25.880627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec858d-8829-4bfd-9271-129627aae0d7 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:38:25.882046 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 550/1100] 127.0.0.1 () {54 vars in 901 bytes} [Sat Oct 15 08:38:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:38:25.888087 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff43c7a-91a8-464c-9801-3564e6a7c92b None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:38:25.888511 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff43c7a-91a8-464c-9801-3564e6a7c92b None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:38:25.888623 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff43c7a-91a8-464c-9801-3564e6a7c92b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:38:25.908987 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:38:25.965437 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-8ff43c7a-91a8-464c-9801-3564e6a7c92b None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:38:25.966685 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-8ff43c7a-91a8-464c-9801-3564e6a7c92b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:38:26.061310 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 551/1101] 127.0.0.1 () {58 vars in 998 bytes} [Sat Oct 15 08:38:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:38:26.209918 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdec858d-8829-4bfd-9271-129627aae0d7 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:38:26.211356 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-fdec858d-8829-4bfd-9271-129627aae0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:38:26.321666 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:38:26.356186 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdec858d-8829-4bfd-9271-129627aae0d7 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:38:26.474346 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a457ea75-c3ae-4ccf-b93d-6309507ca1e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0a8af5cadc4cc1ab73e8eb2a20bb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:38:26.474872 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a457ea75-c3ae-4ccf-b93d-6309507ca1e9 service nova] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:38:26.475150 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a457ea75-c3ae-4ccf-b93d-6309507ca1e9 service nova] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:38:26.475438 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a457ea75-c3ae-4ccf-b93d-6309507ca1e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:38:26.476985 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-a457ea75-c3ae-4ccf-b93d-6309507ca1e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:38:26.552041 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:38:26.565937 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a457ea75-c3ae-4ccf-b93d-6309507ca1e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:38:26.567527 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a457ea75-c3ae-4ccf-b93d-6309507ca1e9 service nova] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:38:26.585003 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 551/1102] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:38:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:38:27.688552 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e188ae85-e0da-48c4-b4ea-51c6be61e150 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:38:27.689079 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e188ae85-e0da-48c4-b4ea-51c6be61e150 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:38:27.689079 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e188ae85-e0da-48c4-b4ea-51c6be61e150 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:38:27.690496 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 552/1103] 127.0.0.1 () {54 vars in 901 bytes} [Sat Oct 15 08:38:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:38:27.696644 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5b523-6fc3-44ef-adc7-62d961c611fb None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:38:27.697105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5b523-6fc3-44ef-adc7-62d961c611fb None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:38:27.697203 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5b523-6fc3-44ef-adc7-62d961c611fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:38:27.772025 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-bca5b523-6fc3-44ef-adc7-62d961c611fb None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:38:27.773232 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-bca5b523-6fc3-44ef-adc7-62d961c611fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:38:27.869041 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 552/1104] 127.0.0.1 () {58 vars in 998 bytes} [Sat Oct 15 08:38:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:38:28.015771 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e188ae85-e0da-48c4-b4ea-51c6be61e150 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:38:28.017275 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-e188ae85-e0da-48c4-b4ea-51c6be61e150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:38:28.100861 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:38:28.162125 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e188ae85-e0da-48c4-b4ea-51c6be61e150 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:38:28.276017 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b8fc7f-e08b-41bd-9fb5-19da3a482a89 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0a8af5cadc4cc1ab73e8eb2a20bb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:38:28.276636 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b8fc7f-e08b-41bd-9fb5-19da3a482a89 service nova] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:38:28.276897 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b8fc7f-e08b-41bd-9fb5-19da3a482a89 service nova] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:38:28.277164 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b8fc7f-e08b-41bd-9fb5-19da3a482a89 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:38:28.278802 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-88b8fc7f-e08b-41bd-9fb5-19da3a482a89 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:38:28.326447 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:38:28.366690 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b8fc7f-e08b-41bd-9fb5-19da3a482a89 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:38:28.368266 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b8fc7f-e08b-41bd-9fb5-19da3a482a89 service nova] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:38:28.386469 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 553/1105] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:38:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:38:29.440155 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df494eb9-e907-4736-ba82-aa47d96dcbb4 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:38:29.440678 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df494eb9-e907-4736-ba82-aa47d96dcbb4 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:38:29.440678 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df494eb9-e907-4736-ba82-aa47d96dcbb4 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:38:29.442080 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 553/1106] 127.0.0.1 () {54 vars in 901 bytes} [Sat Oct 15 08:38:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:38:29.448188 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1026f939-178b-45f1-930a-aabfcd03fcde None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:38:29.448632 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1026f939-178b-45f1-930a-aabfcd03fcde None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:38:29.448730 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1026f939-178b-45f1-930a-aabfcd03fcde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:38:29.522545 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-1026f939-178b-45f1-930a-aabfcd03fcde None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:38:29.523744 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-1026f939-178b-45f1-930a-aabfcd03fcde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:38:29.618445 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 554/1107] 127.0.0.1 () {58 vars in 998 bytes} [Sat Oct 15 08:38:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:38:29.764026 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df494eb9-e907-4736-ba82-aa47d96dcbb4 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:38:29.765548 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-df494eb9-e907-4736-ba82-aa47d96dcbb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:38:29.814075 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:38:29.914999 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df494eb9-e907-4736-ba82-aa47d96dcbb4 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:38:30.029901 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35107824-0654-4d68-bad3-aa4838c6918b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0a8af5cadc4cc1ab73e8eb2a20bb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:38:30.030446 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35107824-0654-4d68-bad3-aa4838c6918b service nova] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:38:30.030704 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35107824-0654-4d68-bad3-aa4838c6918b service nova] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:38:30.030974 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35107824-0654-4d68-bad3-aa4838c6918b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:38:30.032635 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-35107824-0654-4d68-bad3-aa4838c6918b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:38:30.039465 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:38:30.123086 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35107824-0654-4d68-bad3-aa4838c6918b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:38:30.124707 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35107824-0654-4d68-bad3-aa4838c6918b service nova] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:38:30.142897 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 554/1108] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:38:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:38:31.189547 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ca5d24-2281-476f-ae1e-9cba5a2d9226 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:38:31.190187 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ca5d24-2281-476f-ae1e-9cba5a2d9226 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:38:31.190187 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ca5d24-2281-476f-ae1e-9cba5a2d9226 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:38:31.191556 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 555/1109] 127.0.0.1 () {54 vars in 901 bytes} [Sat Oct 15 08:38:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:38:31.197903 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918d1a96-2c89-4249-b444-5f2e5bfc9873 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:38:31.198332 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918d1a96-2c89-4249-b444-5f2e5bfc9873 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:38:31.198468 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918d1a96-2c89-4249-b444-5f2e5bfc9873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:38:31.274395 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-918d1a96-2c89-4249-b444-5f2e5bfc9873 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:38:31.275898 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-918d1a96-2c89-4249-b444-5f2e5bfc9873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:38:31.342152 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:38:31.371384 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 555/1110] 127.0.0.1 () {58 vars in 998 bytes} [Sat Oct 15 08:38:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:38:31.519188 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ca5d24-2281-476f-ae1e-9cba5a2d9226 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:38:31.520664 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-80ca5d24-2281-476f-ae1e-9cba5a2d9226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:38:31.538752 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:38:31.664254 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ca5d24-2281-476f-ae1e-9cba5a2d9226 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:38:31.749908 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:38:31.776246 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770eeda8-9124-47b3-82d5-08906a2ece99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0a8af5cadc4cc1ab73e8eb2a20bb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:38:31.776789 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770eeda8-9124-47b3-82d5-08906a2ece99 service nova] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:38:31.777067 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770eeda8-9124-47b3-82d5-08906a2ece99 service nova] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:38:31.777340 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770eeda8-9124-47b3-82d5-08906a2ece99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:38:31.778961 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-770eeda8-9124-47b3-82d5-08906a2ece99 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:38:31.866053 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770eeda8-9124-47b3-82d5-08906a2ece99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:38:31.867643 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770eeda8-9124-47b3-82d5-08906a2ece99 service nova] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:38:31.885702 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 556/1111] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:38:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:38:32.929670 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f310e191-fe63-4118-9e08-36dbec9709e4 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:38:32.930257 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f310e191-fe63-4118-9e08-36dbec9709e4 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:38:32.930257 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f310e191-fe63-4118-9e08-36dbec9709e4 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:38:32.931596 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 556/1112] 127.0.0.1 () {54 vars in 901 bytes} [Sat Oct 15 08:38:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:38:32.937998 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040776b3-a217-499b-9b20-ea93ebec4f31 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:38:32.938457 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040776b3-a217-499b-9b20-ea93ebec4f31 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:38:32.938590 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040776b3-a217-499b-9b20-ea93ebec4f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:38:33.015288 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-040776b3-a217-499b-9b20-ea93ebec4f31 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:38:33.016493 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-040776b3-a217-499b-9b20-ea93ebec4f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:38:33.058238 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:38:33.112267 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 557/1113] 127.0.0.1 () {58 vars in 998 bytes} [Sat Oct 15 08:38:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:38:33.261691 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f310e191-fe63-4118-9e08-36dbec9709e4 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:38:33.263208 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-f310e191-fe63-4118-9e08-36dbec9709e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:38:33.407660 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f310e191-fe63-4118-9e08-36dbec9709e4 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:38:33.461016 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:38:33.520041 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ab7c8e-3e07-415c-9afe-56a03ec4d2e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0a8af5cadc4cc1ab73e8eb2a20bb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:38:33.520571 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab7c8e-3e07-415c-9afe-56a03ec4d2e6 service nova] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:38:33.520853 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab7c8e-3e07-415c-9afe-56a03ec4d2e6 service nova] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:38:33.521123 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab7c8e-3e07-415c-9afe-56a03ec4d2e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:38:33.522758 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-27ab7c8e-3e07-415c-9afe-56a03ec4d2e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:38:33.611035 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ab7c8e-3e07-415c-9afe-56a03ec4d2e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:38:33.612633 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ab7c8e-3e07-415c-9afe-56a03ec4d2e6 service nova] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:38:33.630567 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 557/1114] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:38:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:38:34.674188 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ac692-eaa8-4c99-ae38-d97d6693bd00 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:38:34.674719 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ac692-eaa8-4c99-ae38-d97d6693bd00 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:38:34.674719 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ac692-eaa8-4c99-ae38-d97d6693bd00 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:38:34.675972 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 558/1115] 127.0.0.1 () {54 vars in 901 bytes} [Sat Oct 15 08:38:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:38:34.682314 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b2ec9d-6615-4246-93a0-d4ea2e36b706 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:38:34.682755 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b2ec9d-6615-4246-93a0-d4ea2e36b706 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:38:34.682854 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b2ec9d-6615-4246-93a0-d4ea2e36b706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:38:34.755965 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-c3b2ec9d-6615-4246-93a0-d4ea2e36b706 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:38:34.757153 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-c3b2ec9d-6615-4246-93a0-d4ea2e36b706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:38:34.759155 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:38:34.853352 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 558/1116] 127.0.0.1 () {58 vars in 998 bytes} [Sat Oct 15 08:38:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:38:35.004159 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119ac692-eaa8-4c99-ae38-d97d6693bd00 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:38:35.005668 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-119ac692-eaa8-4c99-ae38-d97d6693bd00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:38:35.149531 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119ac692-eaa8-4c99-ae38-d97d6693bd00 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:38:35.178675 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:38:35.263803 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b71c66-a531-4c99-bedb-21211b72bee9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0a8af5cadc4cc1ab73e8eb2a20bb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:38:35.264326 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b71c66-a531-4c99-bedb-21211b72bee9 service nova] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:38:35.264603 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b71c66-a531-4c99-bedb-21211b72bee9 service nova] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:38:35.264875 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b71c66-a531-4c99-bedb-21211b72bee9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:38:35.266490 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-08b71c66-a531-4c99-bedb-21211b72bee9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:38:35.354073 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b71c66-a531-4c99-bedb-21211b72bee9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:38:35.355679 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b71c66-a531-4c99-bedb-21211b72bee9 service nova] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:38:35.373724 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 559/1117] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:38:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:38:37.433471 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc3a6b-63a3-456c-a579-c2d3a45c2247 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:38:37.434078 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc3a6b-63a3-456c-a579-c2d3a45c2247 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:38:37.434078 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc3a6b-63a3-456c-a579-c2d3a45c2247 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:38:37.435418 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 559/1118] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 15 08:38:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:38:37.441281 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7561fade-950e-4246-b546-cc755711f1dc None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:38:37.441754 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7561fade-950e-4246-b546-cc755711f1dc None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:38:37.441860 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7561fade-950e-4246-b546-cc755711f1dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:38:37.482188 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:38:37.508189 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-7561fade-950e-4246-b546-cc755711f1dc None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:38:37.509401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-7561fade-950e-4246-b546-cc755711f1dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:38:37.604024 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 560/1119] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 15 08:38:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:38:37.888393 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dc3a6b-63a3-456c-a579-c2d3a45c2247 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:38:37.889965 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-72dc3a6b-63a3-456c-a579-c2d3a45c2247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:38:37.993996 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dc3a6b-63a3-456c-a579-c2d3a45c2247 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:38:38.025831 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:38:38.067579 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff42a171-434e-41dd-9d02-f0ca5eb886f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:38:38.068189 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42a171-434e-41dd-9d02-f0ca5eb886f4 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:38:38.068473 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42a171-434e-41dd-9d02-f0ca5eb886f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:38:38.068752 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42a171-434e-41dd-9d02-f0ca5eb886f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:38:38.070373 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-ff42a171-434e-41dd-9d02-f0ca5eb886f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:38:38.181570 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff42a171-434e-41dd-9d02-f0ca5eb886f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:38:38.183041 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff42a171-434e-41dd-9d02-f0ca5eb886f4 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:38:38.200872 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 560/1120] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:38:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:38:40.481549 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112b18a0-568e-43e2-a5de-b77b203049c2 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:38:40.482130 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112b18a0-568e-43e2-a5de-b77b203049c2 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:38:40.482130 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112b18a0-568e-43e2-a5de-b77b203049c2 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:38:40.483474 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 561/1121] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 15 08:38:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:38:40.489321 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b01e986-8fae-4093-bb76-a281a0fdd437 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:38:40.489751 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b01e986-8fae-4093-bb76-a281a0fdd437 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:38:40.489917 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b01e986-8fae-4093-bb76-a281a0fdd437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:38:40.555339 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-0b01e986-8fae-4093-bb76-a281a0fdd437 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:38:40.556602 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-0b01e986-8fae-4093-bb76-a281a0fdd437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:38:40.569776 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:38:40.654193 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 561/1122] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 15 08:38:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:38:40.937370 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112b18a0-568e-43e2-a5de-b77b203049c2 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:38:40.938946 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-112b18a0-568e-43e2-a5de-b77b203049c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:38:41.041962 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112b18a0-568e-43e2-a5de-b77b203049c2 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:38:41.054937 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:38:41.117648 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1ddeb1-7468-4ad4-b81a-ec8a34c6d373 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:38:41.118277 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ddeb1-7468-4ad4-b81a-ec8a34c6d373 service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:38:41.118561 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ddeb1-7468-4ad4-b81a-ec8a34c6d373 service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:38:41.118843 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ddeb1-7468-4ad4-b81a-ec8a34c6d373 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:38:41.120465 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-ef1ddeb1-7468-4ad4-b81a-ec8a34c6d373 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:38:41.229288 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1ddeb1-7468-4ad4-b81a-ec8a34c6d373 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:38:41.230658 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1ddeb1-7468-4ad4-b81a-ec8a34c6d373 service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:38:41.249578 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 562/1123] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:38:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:38:43.686867 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a922b-e638-4b5d-ac92-0d11d0a46d73 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:38:43.687389 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a922b-e638-4b5d-ac92-0d11d0a46d73 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:38:43.687389 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a922b-e638-4b5d-ac92-0d11d0a46d73 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:38:43.688704 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 562/1124] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 15 08:38:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:38:43.694818 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163df1e2-4918-4308-9611-979301f59a5b None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:38:43.695253 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163df1e2-4918-4308-9611-979301f59a5b None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:38:43.695361 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163df1e2-4918-4308-9611-979301f59a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:38:43.742588 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:38:43.760846 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-163df1e2-4918-4308-9611-979301f59a5b None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:38:43.762067 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-163df1e2-4918-4308-9611-979301f59a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:38:43.856298 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 563/1125] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 15 08:38:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:38:44.139380 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922a922b-e638-4b5d-ac92-0d11d0a46d73 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:38:44.140896 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-922a922b-e638-4b5d-ac92-0d11d0a46d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:38:44.242369 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922a922b-e638-4b5d-ac92-0d11d0a46d73 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:38:44.282389 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:38:44.316003 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3d0a94-ed93-4604-9ddf-e664baa62131 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:38:44.316602 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d0a94-ed93-4604-9ddf-e664baa62131 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:38:44.316877 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d0a94-ed93-4604-9ddf-e664baa62131 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:38:44.317149 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d0a94-ed93-4604-9ddf-e664baa62131 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:38:44.318747 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-2e3d0a94-ed93-4604-9ddf-e664baa62131 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:38:44.428392 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3d0a94-ed93-4604-9ddf-e664baa62131 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:38:44.429842 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3d0a94-ed93-4604-9ddf-e664baa62131 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:38:44.447255 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 563/1126] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:38:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:38:46.720217 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5703c42-3d24-44b6-a5ce-f2559b8d4412 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:38:46.720766 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5703c42-3d24-44b6-a5ce-f2559b8d4412 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:38:46.720766 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5703c42-3d24-44b6-a5ce-f2559b8d4412 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:38:46.722091 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 564/1127] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 15 08:38:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:38:46.727655 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52005c78-6b1a-42d3-bfb8-fccbac7b94d6 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:38:46.728087 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52005c78-6b1a-42d3-bfb8-fccbac7b94d6 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:38:46.728184 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52005c78-6b1a-42d3-bfb8-fccbac7b94d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:38:46.792672 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-52005c78-6b1a-42d3-bfb8-fccbac7b94d6 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:38:46.793920 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-52005c78-6b1a-42d3-bfb8-fccbac7b94d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:38:46.814220 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:38:46.889630 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 564/1128] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 15 08:38:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 08:38:47.175564 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5703c42-3d24-44b6-a5ce-f2559b8d4412 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:38:47.177096 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-e5703c42-3d24-44b6-a5ce-f2559b8d4412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:38:47.278522 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5703c42-3d24-44b6-a5ce-f2559b8d4412 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:38:47.299216 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:38:47.351618 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e426dfb-7b10-4c3a-a2a9-b3fec824c5bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:38:47.352218 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e426dfb-7b10-4c3a-a2a9-b3fec824c5bd service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:38:47.352495 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e426dfb-7b10-4c3a-a2a9-b3fec824c5bd service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:38:47.352767 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e426dfb-7b10-4c3a-a2a9-b3fec824c5bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:38:47.354411 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-9e426dfb-7b10-4c3a-a2a9-b3fec824c5bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:38:47.462353 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e426dfb-7b10-4c3a-a2a9-b3fec824c5bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:38:47.463715 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e426dfb-7b10-4c3a-a2a9-b3fec824c5bd service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:38:47.481623 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 565/1129] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:38:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:38:49.776119 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c41ee-5bea-45fc-ad29-40e88b94cc98 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:38:49.776663 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c41ee-5bea-45fc-ad29-40e88b94cc98 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:38:49.776768 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c41ee-5bea-45fc-ad29-40e88b94cc98 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:38:49.778114 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 565/1130] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 15 08:38:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:38:49.783888 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a93b2-1a67-45bd-a0c0-29cc470b1fa9 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:38:49.784312 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a93b2-1a67-45bd-a0c0-29cc470b1fa9 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:38:49.784444 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a93b2-1a67-45bd-a0c0-29cc470b1fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:38:49.846649 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:38:49.849289 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-7e7a93b2-1a67-45bd-a0c0-29cc470b1fa9 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:38:49.851150 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-7e7a93b2-1a67-45bd-a0c0-29cc470b1fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:38:49.945330 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 566/1131] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 15 08:38:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:38:50.229513 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80c41ee-5bea-45fc-ad29-40e88b94cc98 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:38:50.231062 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-d80c41ee-5bea-45fc-ad29-40e88b94cc98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:38:50.333596 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80c41ee-5bea-45fc-ad29-40e88b94cc98 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:38:50.383423 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:38:50.408818 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b5e644-44d3-4b4e-81b6-f947995cd006 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:38:50.409422 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b5e644-44d3-4b4e-81b6-f947995cd006 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:38:50.409730 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b5e644-44d3-4b4e-81b6-f947995cd006 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:38:50.410007 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b5e644-44d3-4b4e-81b6-f947995cd006 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:38:50.411577 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-b4b5e644-44d3-4b4e-81b6-f947995cd006 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:38:50.523545 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b5e644-44d3-4b4e-81b6-f947995cd006 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:38:50.525255 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b5e644-44d3-4b4e-81b6-f947995cd006 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:38:50.543853 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 566/1132] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:38:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:38:52.721954 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a163f85-b99a-4f9c-a556-27d0a5ffb752 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:38:52.722479 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a163f85-b99a-4f9c-a556-27d0a5ffb752 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:38:52.722479 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a163f85-b99a-4f9c-a556-27d0a5ffb752 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:38:52.723843 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 567/1133] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 15 08:38:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:38:52.729232 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273eedcd-4e2b-47b5-bf30-688f93e3f591 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:38:52.729618 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273eedcd-4e2b-47b5-bf30-688f93e3f591 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:38:52.729809 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273eedcd-4e2b-47b5-bf30-688f93e3f591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:38:52.795039 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-273eedcd-4e2b-47b5-bf30-688f93e3f591 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:38:52.796258 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-273eedcd-4e2b-47b5-bf30-688f93e3f591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:38:52.825766 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:38:52.891927 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 567/1134] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 15 08:38:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:38:53.177906 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a163f85-b99a-4f9c-a556-27d0a5ffb752 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:38:53.179445 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-5a163f85-b99a-4f9c-a556-27d0a5ffb752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:38:53.286055 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a163f85-b99a-4f9c-a556-27d0a5ffb752 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:38:53.317181 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:38:53.362615 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bec404-3d23-4e99-9140-9dd271509b64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:38:53.363225 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bec404-3d23-4e99-9140-9dd271509b64 service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:38:53.363502 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bec404-3d23-4e99-9140-9dd271509b64 service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:38:53.363788 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bec404-3d23-4e99-9140-9dd271509b64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:38:53.365382 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-74bec404-3d23-4e99-9140-9dd271509b64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:38:53.476267 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bec404-3d23-4e99-9140-9dd271509b64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:38:53.477633 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bec404-3d23-4e99-9140-9dd271509b64 service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:38:53.496204 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 568/1135] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:38:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:38:55.691246 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c7bfba-dee0-40aa-ab2e-c5d34a5d9f55 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:38:55.691770 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c7bfba-dee0-40aa-ab2e-c5d34a5d9f55 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:38:55.691770 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c7bfba-dee0-40aa-ab2e-c5d34a5d9f55 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:38:55.693065 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 568/1136] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 15 08:38:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:38:55.698457 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80373ac4-c124-4599-9df0-6c46bca1d893 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:38:55.698923 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80373ac4-c124-4599-9df0-6c46bca1d893 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:38:55.699031 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80373ac4-c124-4599-9df0-6c46bca1d893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:38:55.763824 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-80373ac4-c124-4599-9df0-6c46bca1d893 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:38:55.765030 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-80373ac4-c124-4599-9df0-6c46bca1d893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:38:55.773816 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:38:55.860915 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 569/1137] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 15 08:38:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:38:56.146443 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c7bfba-dee0-40aa-ab2e-c5d34a5d9f55 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:38:56.147990 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-74c7bfba-dee0-40aa-ab2e-c5d34a5d9f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:38:56.250417 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c7bfba-dee0-40aa-ab2e-c5d34a5d9f55 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:38:56.305866 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:38:56.324104 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e5b989-9f06-477c-8935-26712220cb79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:38:56.324709 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e5b989-9f06-477c-8935-26712220cb79 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:38:56.324986 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e5b989-9f06-477c-8935-26712220cb79 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:38:56.325254 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e5b989-9f06-477c-8935-26712220cb79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:38:56.326862 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-76e5b989-9f06-477c-8935-26712220cb79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:38:56.434424 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e5b989-9f06-477c-8935-26712220cb79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:38:56.435844 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e5b989-9f06-477c-8935-26712220cb79 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:38:56.454140 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 569/1138] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:38:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:38:58.648346 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d04137-d1eb-45a4-b9e5-e6029343eaf6 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:38:58.648887 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d04137-d1eb-45a4-b9e5-e6029343eaf6 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:38:58.649022 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d04137-d1eb-45a4-b9e5-e6029343eaf6 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:38:58.650597 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 570/1139] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 15 08:38:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:38:58.655784 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c520a-f926-4b24-bf58-28f56555dc6b None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:38:58.656240 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c520a-f926-4b24-bf58-28f56555dc6b None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:38:58.656347 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c520a-f926-4b24-bf58-28f56555dc6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:38:58.723730 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-680c520a-f926-4b24-bf58-28f56555dc6b None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:38:58.724870 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-680c520a-f926-4b24-bf58-28f56555dc6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:38:58.763784 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:38:58.821957 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 570/1140] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 15 08:38:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:38:59.106144 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d04137-d1eb-45a4-b9e5-e6029343eaf6 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:38:59.107671 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-21d04137-d1eb-45a4-b9e5-e6029343eaf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:38:59.209838 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d04137-d1eb-45a4-b9e5-e6029343eaf6 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:38:59.246756 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:38:59.283022 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbade5e1-5c30-46bb-a728-e5d8c239cd22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:38:59.283623 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbade5e1-5c30-46bb-a728-e5d8c239cd22 service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:38:59.283902 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbade5e1-5c30-46bb-a728-e5d8c239cd22 service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:38:59.284173 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbade5e1-5c30-46bb-a728-e5d8c239cd22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:38:59.285780 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-fbade5e1-5c30-46bb-a728-e5d8c239cd22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:38:59.392918 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbade5e1-5c30-46bb-a728-e5d8c239cd22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:38:59.394341 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbade5e1-5c30-46bb-a728-e5d8c239cd22 service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:38:59.415012 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 571/1141] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:38:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:39:01.571749 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f348516-b763-4b98-9010-803f6f6729e0 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:39:01.572398 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f348516-b763-4b98-9010-803f6f6729e0 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:39:01.572398 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f348516-b763-4b98-9010-803f6f6729e0 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:39:01.573666 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 571/1142] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 15 08:39:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:39:01.579032 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86fb97-9716-4287-9704-cb9ef4b6d163 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:39:01.579992 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86fb97-9716-4287-9704-cb9ef4b6d163 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:39:01.580111 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86fb97-9716-4287-9704-cb9ef4b6d163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:39:01.644562 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-cf86fb97-9716-4287-9704-cb9ef4b6d163 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:39:01.645792 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-cf86fb97-9716-4287-9704-cb9ef4b6d163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:39:01.664508 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:39:01.740323 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 572/1143] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 15 08:39:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:39:02.023630 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f348516-b763-4b98-9010-803f6f6729e0 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:39:02.025464 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-7f348516-b763-4b98-9010-803f6f6729e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:39:02.130014 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f348516-b763-4b98-9010-803f6f6729e0 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:39:02.195618 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:39:02.202779 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaf9d55-f7f8-46ac-8392-f99e324cb382 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:39:02.203378 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf9d55-f7f8-46ac-8392-f99e324cb382 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:39:02.203656 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf9d55-f7f8-46ac-8392-f99e324cb382 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:39:02.203926 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf9d55-f7f8-46ac-8392-f99e324cb382 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:39:02.206076 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-fcaf9d55-f7f8-46ac-8392-f99e324cb382 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:39:02.313782 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaf9d55-f7f8-46ac-8392-f99e324cb382 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:39:02.315135 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaf9d55-f7f8-46ac-8392-f99e324cb382 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:39:02.333063 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 572/1144] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:39:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:39:04.481550 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a9f982-4d41-4e68-8feb-8aca0f36d90d None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:39:04.482140 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a9f982-4d41-4e68-8feb-8aca0f36d90d None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:39:04.482140 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a9f982-4d41-4e68-8feb-8aca0f36d90d None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:39:04.483474 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 573/1145] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 15 08:39:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:39:04.489211 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef106cc-d7ce-43af-a493-eb120df0b6d0 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:39:04.489711 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef106cc-d7ce-43af-a493-eb120df0b6d0 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:39:04.489832 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef106cc-d7ce-43af-a493-eb120df0b6d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:39:04.554988 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-4ef106cc-d7ce-43af-a493-eb120df0b6d0 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:39:04.556289 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-4ef106cc-d7ce-43af-a493-eb120df0b6d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:39:04.604026 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:39:04.652086 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 573/1146] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 15 08:39:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:39:04.936606 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a9f982-4d41-4e68-8feb-8aca0f36d90d None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:39:04.938209 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-b2a9f982-4d41-4e68-8feb-8aca0f36d90d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:39:05.040696 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a9f982-4d41-4e68-8feb-8aca0f36d90d None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:39:05.089139 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:39:05.115384 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e69dd8-1a94-4b63-9137-5b555fcb24e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:39:05.115985 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e69dd8-1a94-4b63-9137-5b555fcb24e2 service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:39:05.116265 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e69dd8-1a94-4b63-9137-5b555fcb24e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:39:05.116537 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e69dd8-1a94-4b63-9137-5b555fcb24e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:39:05.118168 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-85e69dd8-1a94-4b63-9137-5b555fcb24e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:39:05.225142 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e69dd8-1a94-4b63-9137-5b555fcb24e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:39:05.226528 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e69dd8-1a94-4b63-9137-5b555fcb24e2 service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:39:05.245219 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 574/1147] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:39:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:39:14.650580 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=22 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:39:15.242941 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=66 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:39:25.099192 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e21d6-4dc3-45aa-b62b-5030eeaffa5f None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:39:25.099762 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e21d6-4dc3-45aa-b62b-5030eeaffa5f None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:39:25.099868 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e21d6-4dc3-45aa-b62b-5030eeaffa5f None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:39:25.101243 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 574/1148] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:39:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:39:25.106624 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b9e37-56bd-4cea-98c5-7d2070a479a9 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:39:25.107051 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b9e37-56bd-4cea-98c5-7d2070a479a9 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:39:25.107161 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b9e37-56bd-4cea-98c5-7d2070a479a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:39:25.173865 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-677b9e37-56bd-4cea-98c5-7d2070a479a9 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:39:25.174993 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-677b9e37-56bd-4cea-98c5-7d2070a479a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:39:25.269925 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 575/1149] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:39:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:39:25.571504 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6633294b-2309-4042-87b9-29280115678a None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:39:25.571791 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6633294b-2309-4042-87b9-29280115678a None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:39:25.572065 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6633294b-2309-4042-87b9-29280115678a None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:39:25.573408 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 575/1150] 10.222.0.30 () {54 vars in 962 bytes} [Sat Oct 15 08:39:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:39:25.578982 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc095d1-47db-49e6-99ee-f4edbebff6d7 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:39:25.579267 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc095d1-47db-49e6-99ee-f4edbebff6d7 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:39:25.579541 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc095d1-47db-49e6-99ee-f4edbebff6d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:39:25.654287 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-3dc095d1-47db-49e6-99ee-f4edbebff6d7 None None] MFA Rules not processed for user `71692d05fd324679b1598feb52b3fe6d`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:39:25.655470 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-3dc095d1-47db-49e6-99ee-f4edbebff6d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:39:25.675959 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:39:25.730318 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 576/1151] 10.222.0.30 () {58 vars in 1059 bytes} [Sat Oct 15 08:39:25 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:39:25.738531 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6633294b-2309-4042-87b9-29280115678a None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:39:25.740079 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-6633294b-2309-4042-87b9-29280115678a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:39:25.841397 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6633294b-2309-4042-87b9-29280115678a None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:39:25.914714 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1566ab-1609-43cf-87f3-30660f8c95aa service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71692d05fd324679b1598feb52b3fe6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:39:25.915321 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1566ab-1609-43cf-87f3-30660f8c95aa service glance] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:39:25.915598 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1566ab-1609-43cf-87f3-30660f8c95aa service glance] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:39:25.915867 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1566ab-1609-43cf-87f3-30660f8c95aa service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:39:25.917425 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-0c1566ab-1609-43cf-87f3-30660f8c95aa service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:39:25.972480 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:39:26.026486 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1566ab-1609-43cf-87f3-30660f8c95aa service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:39:26.027913 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1566ab-1609-43cf-87f3-30660f8c95aa service glance] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:39:26.045890 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 576/1152] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:39:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:39:28.293637 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a5373-d6bb-4aa0-800d-25298c6dd43b None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:39:28.294222 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a5373-d6bb-4aa0-800d-25298c6dd43b None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:39:28.294222 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a5373-d6bb-4aa0-800d-25298c6dd43b None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:39:28.295604 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 577/1153] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:39:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:39:28.300830 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f11ca7b-f03d-4c2e-b485-1fe4dcdaf62f None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:39:28.301236 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f11ca7b-f03d-4c2e-b485-1fe4dcdaf62f None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:39:28.301419 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f11ca7b-f03d-4c2e-b485-1fe4dcdaf62f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:39:28.365992 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-9f11ca7b-f03d-4c2e-b485-1fe4dcdaf62f None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:39:28.367211 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-9f11ca7b-f03d-4c2e-b485-1fe4dcdaf62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:39:28.464945 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 577/1154] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:39:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:39:28.741199 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50a5373-d6bb-4aa0-800d-25298c6dd43b None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:39:28.742705 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-d50a5373-d6bb-4aa0-800d-25298c6dd43b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:39:28.867486 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50a5373-d6bb-4aa0-800d-25298c6dd43b None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:39:28.917600 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:39:28.960875 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e031477-5a6d-4e93-9308-c53b26833abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:39:28.961476 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e031477-5a6d-4e93-9308-c53b26833abd admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:39:28.961783 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e031477-5a6d-4e93-9308-c53b26833abd admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:39:28.962055 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e031477-5a6d-4e93-9308-c53b26833abd admin admin] PATH_INFO: `/v3/projects` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:39:28.962690 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e031477-5a6d-4e93-9308-c53b26833abd admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:39:28.963798 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: WARNING py.warnings [None req-3e031477-5a6d-4e93-9308-c53b26833abd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 08:39:28.963798 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: warnings.warn(msg) Oct 15 08:39:28.963798 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: Oct 15 08:39:28.964931 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e031477-5a6d-4e93-9308-c53b26833abd admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:39:28.983402 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 578/1155] 10.222.0.30 () {56 vars in 1108 bytes} [Sat Oct 15 08:39:28 2022] GET /identity/v3/projects => generated 2553 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:39:31.164682 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38df7621-bc34-40e9-9d35-375b0c84d378 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:39:31.165234 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38df7621-bc34-40e9-9d35-375b0c84d378 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:39:31.165234 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38df7621-bc34-40e9-9d35-375b0c84d378 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:39:31.166695 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 578/1156] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:39:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:39:31.172624 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82533bed-0dfd-400c-bcc4-686f100cba39 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:39:31.173034 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82533bed-0dfd-400c-bcc4-686f100cba39 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:39:31.173185 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82533bed-0dfd-400c-bcc4-686f100cba39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:39:31.238484 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-82533bed-0dfd-400c-bcc4-686f100cba39 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:39:31.239690 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-82533bed-0dfd-400c-bcc4-686f100cba39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:39:31.334041 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 579/1157] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:39:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:39:31.627047 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38df7621-bc34-40e9-9d35-375b0c84d378 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:39:31.628710 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-38df7621-bc34-40e9-9d35-375b0c84d378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:39:31.638687 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:39:31.772362 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38df7621-bc34-40e9-9d35-375b0c84d378 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:39:31.851477 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:39:31.884085 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4887f331-c576-4026-9670-e90b5f15581c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0a8af5cadc4cc1ab73e8eb2a20bb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:39:31.884613 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4887f331-c576-4026-9670-e90b5f15581c service nova] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:39:31.884893 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4887f331-c576-4026-9670-e90b5f15581c service nova] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:39:31.885166 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4887f331-c576-4026-9670-e90b5f15581c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:39:31.886819 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-4887f331-c576-4026-9670-e90b5f15581c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:39:31.975746 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4887f331-c576-4026-9670-e90b5f15581c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:39:31.977358 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4887f331-c576-4026-9670-e90b5f15581c service nova] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:39:31.995409 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 579/1158] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:39:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:39:34.300039 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6385fafd-7e95-4f20-8756-75289841b3af None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:39:34.300622 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6385fafd-7e95-4f20-8756-75289841b3af None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:39:34.300622 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6385fafd-7e95-4f20-8756-75289841b3af None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:39:34.301991 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 580/1159] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:39:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:39:34.307456 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fbb1da-7698-4ce0-877d-c2568b4e55db None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:39:34.307901 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fbb1da-7698-4ce0-877d-c2568b4e55db None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:39:34.308013 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fbb1da-7698-4ce0-877d-c2568b4e55db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:39:34.371743 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-b8fbb1da-7698-4ce0-877d-c2568b4e55db None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:39:34.372936 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-b8fbb1da-7698-4ce0-877d-c2568b4e55db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:39:34.418535 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:39:34.469098 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 580/1160] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:39:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:39:34.757823 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6385fafd-7e95-4f20-8756-75289841b3af None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:39:34.759414 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-6385fafd-7e95-4f20-8756-75289841b3af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:39:34.784046 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:39:34.862986 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6385fafd-7e95-4f20-8756-75289841b3af None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:39:34.935915 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba81f3b-e203-4cb8-85b2-bbddbc480c47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:39:34.936519 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba81f3b-e203-4cb8-85b2-bbddbc480c47 service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:39:34.937333 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba81f3b-e203-4cb8-85b2-bbddbc480c47 service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:39:34.937605 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba81f3b-e203-4cb8-85b2-bbddbc480c47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:39:34.939229 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-aba81f3b-e203-4cb8-85b2-bbddbc480c47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:39:35.012102 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:39:35.049284 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba81f3b-e203-4cb8-85b2-bbddbc480c47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:39:35.050745 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba81f3b-e203-4cb8-85b2-bbddbc480c47 service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:39:35.068083 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 581/1161] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:39:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:39:37.256439 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af4c3da-bdf0-4ff9-9247-0ce16a96bee1 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:39:37.257042 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af4c3da-bdf0-4ff9-9247-0ce16a96bee1 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:39:37.257042 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af4c3da-bdf0-4ff9-9247-0ce16a96bee1 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:39:37.258387 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 581/1162] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:39:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:39:37.263839 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9639e6d1-5517-4d24-891b-23f17790cc45 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:39:37.264254 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9639e6d1-5517-4d24-891b-23f17790cc45 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:39:37.264423 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9639e6d1-5517-4d24-891b-23f17790cc45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:39:37.330879 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-9639e6d1-5517-4d24-891b-23f17790cc45 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:39:37.332062 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-9639e6d1-5517-4d24-891b-23f17790cc45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:39:37.427514 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 582/1163] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:39:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:39:37.713179 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af4c3da-bdf0-4ff9-9247-0ce16a96bee1 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:39:37.714743 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-2af4c3da-bdf0-4ff9-9247-0ce16a96bee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:39:37.816648 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af4c3da-bdf0-4ff9-9247-0ce16a96bee1 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:39:37.887751 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:39:37.889211 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd261b3-0e7f-4c52-868a-4e0f311d9860 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:39:37.889984 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd261b3-0e7f-4c52-868a-4e0f311d9860 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:39:37.890456 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd261b3-0e7f-4c52-868a-4e0f311d9860 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:39:37.890894 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd261b3-0e7f-4c52-868a-4e0f311d9860 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:39:37.892716 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-ffd261b3-0e7f-4c52-868a-4e0f311d9860 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:39:38.000974 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd261b3-0e7f-4c52-868a-4e0f311d9860 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:39:38.002435 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd261b3-0e7f-4c52-868a-4e0f311d9860 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:39:38.019939 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 582/1164] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:39:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:39:40.235456 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b99ee5-4b1b-406d-9e78-bada4343c236 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:39:40.235985 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b99ee5-4b1b-406d-9e78-bada4343c236 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:39:40.235985 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b99ee5-4b1b-406d-9e78-bada4343c236 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:39:40.237273 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 583/1165] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:39:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:39:40.242515 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdadb99-aef1-47db-b809-f0a7e0049c1d None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:39:40.242925 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdadb99-aef1-47db-b809-f0a7e0049c1d None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:39:40.243066 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdadb99-aef1-47db-b809-f0a7e0049c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:39:40.307433 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-dcdadb99-aef1-47db-b809-f0a7e0049c1d None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:39:40.308628 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-dcdadb99-aef1-47db-b809-f0a7e0049c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:39:40.360842 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:39:40.403906 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 583/1166] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:39:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:39:40.696031 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b99ee5-4b1b-406d-9e78-bada4343c236 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:39:40.697604 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-29b99ee5-4b1b-406d-9e78-bada4343c236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:39:40.733701 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:39:40.799645 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b99ee5-4b1b-406d-9e78-bada4343c236 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:39:40.871533 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298f579b-de54-44bd-b9ec-6722dec27ba6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:39:40.872129 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f579b-de54-44bd-b9ec-6722dec27ba6 service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:39:40.872435 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f579b-de54-44bd-b9ec-6722dec27ba6 service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:39:40.872704 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f579b-de54-44bd-b9ec-6722dec27ba6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:39:40.874874 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-298f579b-de54-44bd-b9ec-6722dec27ba6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:39:40.954587 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:39:40.983344 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298f579b-de54-44bd-b9ec-6722dec27ba6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:39:40.984778 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298f579b-de54-44bd-b9ec-6722dec27ba6 service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:39:41.002807 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 584/1167] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:39:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:39:43.324826 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30655329-24fa-4b20-9fa3-2df93073cbdd None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:39:43.325398 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30655329-24fa-4b20-9fa3-2df93073cbdd None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:39:43.325398 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30655329-24fa-4b20-9fa3-2df93073cbdd None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:39:43.327451 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 584/1168] 10.222.0.30 () {54 vars in 927 bytes} [Sat Oct 15 08:39:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:39:43.332814 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9803fa-8fdc-4650-90ec-e4d7cd2e1b4e None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:39:43.333222 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9803fa-8fdc-4650-90ec-e4d7cd2e1b4e None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:39:43.333362 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9803fa-8fdc-4650-90ec-e4d7cd2e1b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:39:43.399844 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-7d9803fa-8fdc-4650-90ec-e4d7cd2e1b4e None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:39:43.400455 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-7d9803fa-8fdc-4650-90ec-e4d7cd2e1b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:39:43.494456 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 585/1169] 10.222.0.30 () {58 vars in 1024 bytes} [Sat Oct 15 08:39:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:39:43.783258 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30655329-24fa-4b20-9fa3-2df93073cbdd None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:39:43.784790 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-30655329-24fa-4b20-9fa3-2df93073cbdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:39:43.886180 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30655329-24fa-4b20-9fa3-2df93073cbdd None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:39:43.957707 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae1141e-34c7-4a16-8c6b-00ab7790fc2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:39:43.958326 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae1141e-34c7-4a16-8c6b-00ab7790fc2c service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:39:43.958606 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae1141e-34c7-4a16-8c6b-00ab7790fc2c service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:39:43.958881 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae1141e-34c7-4a16-8c6b-00ab7790fc2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:39:43.960498 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-0ae1141e-34c7-4a16-8c6b-00ab7790fc2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:39:43.969449 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:39:44.069636 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae1141e-34c7-4a16-8c6b-00ab7790fc2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:39:44.071129 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae1141e-34c7-4a16-8c6b-00ab7790fc2c service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:39:44.089876 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 585/1170] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:39:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:39:53.492905 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=82 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:39:54.087617 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=50 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:41:03.179720 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e459295-cbcd-40a1-9936-5f54af601aa2 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:41:03.180338 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e459295-cbcd-40a1-9936-5f54af601aa2 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:41:03.180338 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e459295-cbcd-40a1-9936-5f54af601aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:41:03.256541 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-5e459295-cbcd-40a1-9936-5f54af601aa2 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:41:03.257838 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-5e459295-cbcd-40a1-9936-5f54af601aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:41:03.352744 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 586/1171] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:41:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:41:03.360783 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01f17bb-c924-4061-b17b-f8069588eda0 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:41:03.361301 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01f17bb-c924-4061-b17b-f8069588eda0 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:41:03.361405 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01f17bb-c924-4061-b17b-f8069588eda0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:41:03.437994 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-b01f17bb-c924-4061-b17b-f8069588eda0 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:41:03.439150 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-b01f17bb-c924-4061-b17b-f8069588eda0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:41:03.533850 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 586/1172] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:41:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:41:03.538468 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e459295-cbcd-40a1-9936-5f54af601aa2 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:41:03.539673 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-5e459295-cbcd-40a1-9936-5f54af601aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:41:03.551418 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e459295-cbcd-40a1-9936-5f54af601aa2 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:41:03.611916 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:41:03.645881 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8304fcf-d27f-4ef7-8406-bed03bc156b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:41:03.646487 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8304fcf-d27f-4ef7-8406-bed03bc156b7 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:41:03.646766 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8304fcf-d27f-4ef7-8406-bed03bc156b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:41:03.647036 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8304fcf-d27f-4ef7-8406-bed03bc156b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:41:03.647677 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8304fcf-d27f-4ef7-8406-bed03bc156b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:41:03.648232 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8304fcf-d27f-4ef7-8406-bed03bc156b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:41:03.649249 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: WARNING py.warnings [None req-b8304fcf-d27f-4ef7-8406-bed03bc156b7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 08:41:03.649249 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: warnings.warn(msg) Oct 15 08:41:03.649249 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: Oct 15 08:41:03.650162 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8304fcf-d27f-4ef7-8406-bed03bc156b7 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:41:03.661551 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 587/1173] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 08:41:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:41:03.671499 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01f17bb-c924-4061-b17b-f8069588eda0 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:41:03.682525 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01f17bb-c924-4061-b17b-f8069588eda0 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:41:03.744216 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:41:03.777584 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1212b37-ffec-4b92-96ba-bc4374c391c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:41:03.778211 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1212b37-ffec-4b92-96ba-bc4374c391c4 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:41:03.778493 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1212b37-ffec-4b92-96ba-bc4374c391c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:41:03.778762 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1212b37-ffec-4b92-96ba-bc4374c391c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:41:03.779390 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1212b37-ffec-4b92-96ba-bc4374c391c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:41:03.780483 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: WARNING py.warnings [None req-e1212b37-ffec-4b92-96ba-bc4374c391c4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 08:41:03.780483 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: warnings.warn(msg) Oct 15 08:41:03.780483 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: Oct 15 08:41:03.781670 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1212b37-ffec-4b92-96ba-bc4374c391c4 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:41:03.839060 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 587/1174] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 08:41:03 2022] POST /identity/v3/projects => generated 377 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:41:03.843262 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8304fcf-d27f-4ef7-8406-bed03bc156b7 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:41:03.844789 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-b8304fcf-d27f-4ef7-8406-bed03bc156b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:41:03.877525 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8304fcf-d27f-4ef7-8406-bed03bc156b7 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:41:03.970603 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fac9f1f-8f59-481b-9b67-228e526d5ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:41:03.971213 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac9f1f-8f59-481b-9b67-228e526d5ae3 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:41:03.971497 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac9f1f-8f59-481b-9b67-228e526d5ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:41:03.971768 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac9f1f-8f59-481b-9b67-228e526d5ae3 admin admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:41:03.972407 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fac9f1f-8f59-481b-9b67-228e526d5ae3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:41:03.973458 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: WARNING py.warnings [None req-0fac9f1f-8f59-481b-9b67-228e526d5ae3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 08:41:03.973458 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: warnings.warn(msg) Oct 15 08:41:03.973458 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: Oct 15 08:41:03.974696 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fac9f1f-8f59-481b-9b67-228e526d5ae3 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:41:04.016293 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 588/1175] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:41:03 2022] POST /identity/v3/users => generated 289 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:41:04.020396 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1212b37-ffec-4b92-96ba-bc4374c391c4 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:41:04.021936 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-e1212b37-ffec-4b92-96ba-bc4374c391c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:41:04.083963 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1212b37-ffec-4b92-96ba-bc4374c391c4 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:41:04.166637 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:41:04.178586 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a97b14b-d47b-4a9c-8699-8bca4d599c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:41:04.179189 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a97b14b-d47b-4a9c-8699-8bca4d599c6d admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:41:04.179467 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a97b14b-d47b-4a9c-8699-8bca4d599c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:41:04.179736 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a97b14b-d47b-4a9c-8699-8bca4d599c6d admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:41:04.180367 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a97b14b-d47b-4a9c-8699-8bca4d599c6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:41:04.182021 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a97b14b-d47b-4a9c-8699-8bca4d599c6d admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:41:04.194613 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 588/1176] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:41:04 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:41:04.199265 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fac9f1f-8f59-481b-9b67-228e526d5ae3 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:41:04.200794 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-0fac9f1f-8f59-481b-9b67-228e526d5ae3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:41:04.212124 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fac9f1f-8f59-481b-9b67-228e526d5ae3 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:41:04.236632 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:41:04.307737 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f81ce7d-4768-4d81-9e90-c497cdbf0711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:41:04.308270 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81ce7d-4768-4d81-9e90-c497cdbf0711 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:41:04.308550 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81ce7d-4768-4d81-9e90-c497cdbf0711 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:41:04.308821 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81ce7d-4768-4d81-9e90-c497cdbf0711 admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:41:04.309459 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f81ce7d-4768-4d81-9e90-c497cdbf0711 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:41:04.311107 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f81ce7d-4768-4d81-9e90-c497cdbf0711 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:41:04.323899 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 589/1177] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:41:04 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:41:04.328041 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a97b14b-d47b-4a9c-8699-8bca4d599c6d admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:41:04.329648 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-4a97b14b-d47b-4a9c-8699-8bca4d599c6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:41:04.362573 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a97b14b-d47b-4a9c-8699-8bca4d599c6d admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:41:04.458191 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a79b2d-346b-4aa1-ad02-6876adafb620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:41:04.458864 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a79b2d-346b-4aa1-ad02-6876adafb620 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:41:04.459148 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a79b2d-346b-4aa1-ad02-6876adafb620 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:41:04.459428 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a79b2d-346b-4aa1-ad02-6876adafb620 admin admin] PATH_INFO: `/v3/projects/ffd29518d2fe4ee19763bdf046727f79/users/d25a6b989aa442dc87093a694ec6b35d/roles/3248f0268738489b883041851d681bb7` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:41:04.482681 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a79b2d-346b-4aa1-ad02-6876adafb620 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffd29518d2fe4ee19763bdf046727f79, user_id=d25a6b989aa442dc87093a694ec6b35d, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:41:04.485300 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a79b2d-346b-4aa1-ad02-6876adafb620 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:41:04.509626 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 589/1178] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:41:04 2022] PUT /identity/v3/projects/ffd29518d2fe4ee19763bdf046727f79/users/d25a6b989aa442dc87093a694ec6b35d/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:41:04.520710 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8be159-05f9-423f-afa6-786858cef8ef None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:41:04.521112 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8be159-05f9-423f-afa6-786858cef8ef None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:41:04.521499 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8be159-05f9-423f-afa6-786858cef8ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:41:04.577172 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-0b8be159-05f9-423f-afa6-786858cef8ef None None] MFA Rules not processed for user `d25a6b989aa442dc87093a694ec6b35d`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:41:04.578320 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-0b8be159-05f9-423f-afa6-786858cef8ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:41:04.654799 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 590/1179] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:41:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:41:04.668607 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3fe5c-e100-4dde-b1ad-853867420959 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:41:04.668918 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3fe5c-e100-4dde-b1ad-853867420959 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:41:04.669200 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3fe5c-e100-4dde-b1ad-853867420959 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:41:04.670581 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 590/1180] 10.222.0.30 () {54 vars in 963 bytes} [Sat Oct 15 08:41:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:41:04.675297 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34919810-860f-495d-89ab-9dd7a02f3be2 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:41:04.675651 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34919810-860f-495d-89ab-9dd7a02f3be2 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:41:04.675921 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34919810-860f-495d-89ab-9dd7a02f3be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:41:04.707402 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:41:04.753511 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-34919810-860f-495d-89ab-9dd7a02f3be2 None None] MFA Rules not processed for user `a649c5fedad847fb9e784e779da3a24e`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:41:04.754749 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-34919810-860f-495d-89ab-9dd7a02f3be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:41:04.829432 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 591/1181] 10.222.0.30 () {58 vars in 1060 bytes} [Sat Oct 15 08:41:04 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:41:04.837017 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e3fe5c-e100-4dde-b1ad-853867420959 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:41:04.839167 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-c1e3fe5c-e100-4dde-b1ad-853867420959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:41:04.888202 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:41:04.942304 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e3fe5c-e100-4dde-b1ad-853867420959 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:41:05.014134 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec78f7d-0bfc-4ac0-a226-50d934ba08e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:41:05.014738 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec78f7d-0bfc-4ac0-a226-50d934ba08e5 service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:41:05.015022 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec78f7d-0bfc-4ac0-a226-50d934ba08e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:41:05.015299 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec78f7d-0bfc-4ac0-a226-50d934ba08e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:41:05.016888 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-eec78f7d-0bfc-4ac0-a226-50d934ba08e5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:41:05.121755 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec78f7d-0bfc-4ac0-a226-50d934ba08e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:41:05.123137 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec78f7d-0bfc-4ac0-a226-50d934ba08e5 service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:41:05.126566 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:41:05.141019 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 591/1182] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:41:04 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:41:05.188256 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ef6691-5abb-47e8-85d1-7079d68b4b66 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:41:05.188546 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ef6691-5abb-47e8-85d1-7079d68b4b66 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:41:05.188828 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ef6691-5abb-47e8-85d1-7079d68b4b66 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:41:05.190135 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 592/1183] 10.222.0.30 () {52 vars in 830 bytes} [Sat Oct 15 08:41:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:41:05.193731 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec78f7d-0bfc-4ac0-a226-50d934ba08e5 service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:41:05.194971 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-eec78f7d-0bfc-4ac0-a226-50d934ba08e5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:41:05.295166 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec78f7d-0bfc-4ac0-a226-50d934ba08e5 service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:41:05.387616 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd40338-e028-4148-9a15-0314987aeed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:41:05.388231 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd40338-e028-4148-9a15-0314987aeed2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:41:05.388457 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd40338-e028-4148-9a15-0314987aeed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:41:05.388733 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd40338-e028-4148-9a15-0314987aeed2 admin admin] PATH_INFO: `/v3/users/d25a6b989aa442dc87093a694ec6b35d` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:41:05.402978 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd40338-e028-4148-9a15-0314987aeed2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d25a6b989aa442dc87093a694ec6b35d)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:41:05.404009 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: WARNING py.warnings [None req-edd40338-e028-4148-9a15-0314987aeed2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 08:41:05.404009 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: warnings.warn(msg) Oct 15 08:41:05.404009 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: Oct 15 08:41:05.405253 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd40338-e028-4148-9a15-0314987aeed2 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:41:05.415047 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:41:05.468889 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-edd40338-e028-4148-9a15-0314987aeed2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd25a6b989aa442dc87093a694ec6b35d'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:41:05.469344 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-edd40338-e028-4148-9a15-0314987aeed2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd25a6b989aa442dc87093a694ec6b35d'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:41:05.482840 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-edd40338-e028-4148-9a15-0314987aeed2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd25a6b989aa442dc87093a694ec6b35d'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:41:05.504986 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-edd40338-e028-4148-9a15-0314987aeed2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd25a6b989aa442dc87093a694ec6b35d'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:41:05.514358 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-edd40338-e028-4148-9a15-0314987aeed2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd25a6b989aa442dc87093a694ec6b35d'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:41:05.520285 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 592/1184] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 08:41:05 2022] DELETE /identity/v3/users/d25a6b989aa442dc87093a694ec6b35d => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:41:05.524946 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93908809-e969-4b8e-addd-690a5a385bd8 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:41:05.525428 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93908809-e969-4b8e-addd-690a5a385bd8 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:41:05.525534 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93908809-e969-4b8e-addd-690a5a385bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:41:05.582883 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-93908809-e969-4b8e-addd-690a5a385bd8 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:41:05.584068 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-93908809-e969-4b8e-addd-690a5a385bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:41:05.658863 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:41:05.671856 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 593/1185] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:41:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:41:05.682213 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd40338-e028-4148-9a15-0314987aeed2 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:41:05.683428 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-edd40338-e028-4148-9a15-0314987aeed2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:41:05.782584 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd40338-e028-4148-9a15-0314987aeed2 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:41:05.835965 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:41:05.856640 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce5b894-216a-488b-9f85-d326eb0203fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:41:05.857261 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce5b894-216a-488b-9f85-d326eb0203fb service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:41:05.857542 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce5b894-216a-488b-9f85-d326eb0203fb service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:41:05.857855 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce5b894-216a-488b-9f85-d326eb0203fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:41:05.859449 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-fce5b894-216a-488b-9f85-d326eb0203fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:41:05.966593 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce5b894-216a-488b-9f85-d326eb0203fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:41:05.967958 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce5b894-216a-488b-9f85-d326eb0203fb service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:41:05.986279 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 593/1186] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:41:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:41:06.490875 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93908809-e969-4b8e-addd-690a5a385bd8 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:41:06.492216 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-93908809-e969-4b8e-addd-690a5a385bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:41:06.504168 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93908809-e969-4b8e-addd-690a5a385bd8 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:41:06.603912 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2ead11-5562-4d11-bc08-c3df4ed44804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:41:06.605695 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2ead11-5562-4d11-bc08-c3df4ed44804 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:41:06.606085 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2ead11-5562-4d11-bc08-c3df4ed44804 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:41:06.606462 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2ead11-5562-4d11-bc08-c3df4ed44804 admin admin] PATH_INFO: `/v3/projects/ffd29518d2fe4ee19763bdf046727f79` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:41:06.617959 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2ead11-5562-4d11-bc08-c3df4ed44804 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffd29518d2fe4ee19763bdf046727f79)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:41:06.618999 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: WARNING py.warnings [None req-0d2ead11-5562-4d11-bc08-c3df4ed44804 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 08:41:06.618999 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: warnings.warn(msg) Oct 15 08:41:06.618999 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: Oct 15 08:41:06.620418 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2ead11-5562-4d11-bc08-c3df4ed44804 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:41:06.676885 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-0d2ead11-5562-4d11-bc08-c3df4ed44804 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffd29518d2fe4ee19763bdf046727f79', 'request_id': 'req-0d2ead11-5562-4d11-bc08-c3df4ed44804'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:41:06.690499 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-0d2ead11-5562-4d11-bc08-c3df4ed44804 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffd29518d2fe4ee19763bdf046727f79', 'request_id': 'req-0d2ead11-5562-4d11-bc08-c3df4ed44804'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:41:06.700180 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-0d2ead11-5562-4d11-bc08-c3df4ed44804 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffd29518d2fe4ee19763bdf046727f79', 'request_id': 'req-0d2ead11-5562-4d11-bc08-c3df4ed44804'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:41:06.724511 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-0d2ead11-5562-4d11-bc08-c3df4ed44804 admin admin] The token cache is being invalidate because project ffd29518d2fe4ee19763bdf046727f79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 08:41:06.724799 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-0d2ead11-5562-4d11-bc08-c3df4ed44804 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:41:06.725307 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-0d2ead11-5562-4d11-bc08-c3df4ed44804 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:41:06.726620 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 594/1187] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 08:41:06 2022] DELETE /identity/v3/projects/ffd29518d2fe4ee19763bdf046727f79 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:41:15.984041 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=62 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:41:16.722086 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:42:31.785456 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78db9373-7d4f-4584-936b-5d51fa2523cc None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:42:31.786064 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78db9373-7d4f-4584-936b-5d51fa2523cc None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:42:31.786064 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78db9373-7d4f-4584-936b-5d51fa2523cc None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:42:31.787420 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 594/1188] 127.0.0.1 () {54 vars in 901 bytes} [Sat Oct 15 08:42:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:42:31.792182 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc7554-1a04-4043-929d-8f5d3a6fea77 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:42:31.792620 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc7554-1a04-4043-929d-8f5d3a6fea77 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:42:31.792716 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc7554-1a04-4043-929d-8f5d3a6fea77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:42:31.872619 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-38bc7554-1a04-4043-929d-8f5d3a6fea77 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:42:31.873866 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-38bc7554-1a04-4043-929d-8f5d3a6fea77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:42:31.973920 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 595/1189] 127.0.0.1 () {58 vars in 998 bytes} [Sat Oct 15 08:42:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:42:32.130007 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78db9373-7d4f-4584-936b-5d51fa2523cc None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:42:32.132104 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-78db9373-7d4f-4584-936b-5d51fa2523cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:42:32.249367 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78db9373-7d4f-4584-936b-5d51fa2523cc None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:42:32.254817 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6582a9f-d20d-463b-a2b6-7b01f0585fcf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0a8af5cadc4cc1ab73e8eb2a20bb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:42:32.255421 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6582a9f-d20d-463b-a2b6-7b01f0585fcf service nova] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:42:32.255708 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6582a9f-d20d-463b-a2b6-7b01f0585fcf service nova] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:42:32.255986 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6582a9f-d20d-463b-a2b6-7b01f0585fcf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:42:32.263956 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6582a9f-d20d-463b-a2b6-7b01f0585fcf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:42:32.265542 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6582a9f-d20d-463b-a2b6-7b01f0585fcf service nova] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:42:32.275892 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 595/1190] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:42:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:42:33.359817 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1e9064-9475-4064-b479-3f8a960e79f6 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:42:33.360375 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1e9064-9475-4064-b479-3f8a960e79f6 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:42:33.360484 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1e9064-9475-4064-b479-3f8a960e79f6 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:42:33.361779 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 596/1191] 127.0.0.1 () {54 vars in 901 bytes} [Sat Oct 15 08:42:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 08:42:33.368755 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3635d40d-ad50-4124-8d23-ef3f98daae01 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:42:33.369211 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3635d40d-ad50-4124-8d23-ef3f98daae01 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:42:33.369318 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3635d40d-ad50-4124-8d23-ef3f98daae01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:42:33.388494 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-3635d40d-ad50-4124-8d23-ef3f98daae01 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:42:33.389712 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-3635d40d-ad50-4124-8d23-ef3f98daae01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:42:33.398234 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 596/1192] 127.0.0.1 () {58 vars in 998 bytes} [Sat Oct 15 08:42:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:42:33.549748 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1e9064-9475-4064-b479-3f8a960e79f6 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:42:33.564725 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1e9064-9475-4064-b479-3f8a960e79f6 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:42:33.570169 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37a2d6c-7770-4db0-ade7-82209d5a5d57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0a8af5cadc4cc1ab73e8eb2a20bb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:42:33.570768 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37a2d6c-7770-4db0-ade7-82209d5a5d57 service nova] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:42:33.571052 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37a2d6c-7770-4db0-ade7-82209d5a5d57 service nova] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:42:33.571330 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37a2d6c-7770-4db0-ade7-82209d5a5d57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:42:33.579255 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37a2d6c-7770-4db0-ade7-82209d5a5d57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:42:33.580895 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37a2d6c-7770-4db0-ade7-82209d5a5d57 service nova] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:42:33.590257 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 597/1193] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:42:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:42:33.791620 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede167e-713e-406c-a20b-08c90249f76b None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:42:33.791909 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede167e-713e-406c-a20b-08c90249f76b None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:42:33.792188 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede167e-713e-406c-a20b-08c90249f76b None None] PATH_INFO: `/v3` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:42:33.793480 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 597/1194] 10.222.0.33 () {54 vars in 925 bytes} [Sat Oct 15 08:42:33 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:42:33.823190 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f56dba5-b9e9-471c-9403-4afe61cb4c14 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:42:33.823423 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f56dba5-b9e9-471c-9403-4afe61cb4c14 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:42:33.823701 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f56dba5-b9e9-471c-9403-4afe61cb4c14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:42:33.881145 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-0f56dba5-b9e9-471c-9403-4afe61cb4c14 None None] MFA Rules not processed for user `38986ca2612c4fc59d960cd693834f9b`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:42:33.882355 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-0f56dba5-b9e9-471c-9403-4afe61cb4c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:42:33.899085 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:42:33.952028 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 598/1195] 10.222.0.33 () {58 vars in 1015 bytes} [Sat Oct 15 08:42:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:42:37.417450 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc6da06-4ba8-4442-9454-bb8f617c9d7b None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:42:37.417970 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc6da06-4ba8-4442-9454-bb8f617c9d7b None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:42:37.418075 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc6da06-4ba8-4442-9454-bb8f617c9d7b None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:42:37.419404 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 598/1196] 10.222.0.30 () {54 vars in 961 bytes} [Sat Oct 15 08:42:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:42:37.424293 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ee6d79-728d-47d8-a7a7-3a9c368daa1c None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:42:37.424728 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ee6d79-728d-47d8-a7a7-3a9c368daa1c None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:42:37.424840 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ee6d79-728d-47d8-a7a7-3a9c368daa1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:42:37.444353 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-a8ee6d79-728d-47d8-a7a7-3a9c368daa1c None None] MFA Rules not processed for user `38986ca2612c4fc59d960cd693834f9b`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:42:37.445543 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-a8ee6d79-728d-47d8-a7a7-3a9c368daa1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:42:37.454590 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 599/1197] 10.222.0.30 () {58 vars in 1058 bytes} [Sat Oct 15 08:42:37 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:42:37.462024 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc6da06-4ba8-4442-9454-bb8f617c9d7b None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:42:37.475164 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc6da06-4ba8-4442-9454-bb8f617c9d7b None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:42:37.480192 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668e8a68-29f0-4c6d-9ad0-d961aa2d711f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38986ca2612c4fc59d960cd693834f9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:42:37.480785 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e8a68-29f0-4c6d-9ad0-d961aa2d711f service placement] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:42:37.481071 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e8a68-29f0-4c6d-9ad0-d961aa2d711f service placement] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:42:37.481349 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e8a68-29f0-4c6d-9ad0-d961aa2d711f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:42:37.489418 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668e8a68-29f0-4c6d-9ad0-d961aa2d711f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:42:37.491036 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668e8a68-29f0-4c6d-9ad0-d961aa2d711f service placement] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:42:37.500944 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 599/1198] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:42:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:42:45.049983 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a9a87f-a33e-40bd-8881-5682238a4211 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:42:45.050552 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a9a87f-a33e-40bd-8881-5682238a4211 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:42:45.050552 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a9a87f-a33e-40bd-8881-5682238a4211 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:42:45.051886 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 600/1199] 127.0.0.1 () {54 vars in 901 bytes} [Sat Oct 15 08:42:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:42:45.058660 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525151bc-c0e1-43f0-82c9-87acfaac0d36 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:42:45.059083 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525151bc-c0e1-43f0-82c9-87acfaac0d36 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:42:45.059253 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525151bc-c0e1-43f0-82c9-87acfaac0d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:42:45.078802 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-525151bc-c0e1-43f0-82c9-87acfaac0d36 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:42:45.080002 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-525151bc-c0e1-43f0-82c9-87acfaac0d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:42:45.089113 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 600/1200] 127.0.0.1 () {58 vars in 998 bytes} [Sat Oct 15 08:42:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 08:42:45.242079 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a9a87f-a33e-40bd-8881-5682238a4211 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:42:45.256714 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a9a87f-a33e-40bd-8881-5682238a4211 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:42:45.262663 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522f0e42-ef54-4460-aff6-1488e7efe461 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0a8af5cadc4cc1ab73e8eb2a20bb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:42:45.263256 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522f0e42-ef54-4460-aff6-1488e7efe461 service nova] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:42:45.263535 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522f0e42-ef54-4460-aff6-1488e7efe461 service nova] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:42:45.263809 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522f0e42-ef54-4460-aff6-1488e7efe461 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:42:45.271558 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522f0e42-ef54-4460-aff6-1488e7efe461 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:42:45.273390 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522f0e42-ef54-4460-aff6-1488e7efe461 service nova] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:42:45.282460 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 601/1201] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:42:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:42:46.349487 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf3b28-5d75-4944-a0ff-f105b8ad45f9 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:42:46.350062 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf3b28-5d75-4944-a0ff-f105b8ad45f9 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:42:46.350062 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf3b28-5d75-4944-a0ff-f105b8ad45f9 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:42:46.351329 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 601/1202] 127.0.0.1 () {54 vars in 901 bytes} [Sat Oct 15 08:42:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:42:46.357514 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d56b9f-1da7-4344-b518-3198677f6f8c None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:42:46.358009 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d56b9f-1da7-4344-b518-3198677f6f8c None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:42:46.358119 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d56b9f-1da7-4344-b518-3198677f6f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:42:46.377405 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-18d56b9f-1da7-4344-b518-3198677f6f8c None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:42:46.378641 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-18d56b9f-1da7-4344-b518-3198677f6f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:42:46.387062 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 602/1203] 127.0.0.1 () {58 vars in 998 bytes} [Sat Oct 15 08:42:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:42:46.538650 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bf3b28-5d75-4944-a0ff-f105b8ad45f9 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:42:46.552746 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bf3b28-5d75-4944-a0ff-f105b8ad45f9 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:42:46.558718 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8daf764-2110-4000-a671-38f63f56687f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0a8af5cadc4cc1ab73e8eb2a20bb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:42:46.559310 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8daf764-2110-4000-a671-38f63f56687f service nova] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:42:46.559597 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8daf764-2110-4000-a671-38f63f56687f service nova] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:42:46.559877 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8daf764-2110-4000-a671-38f63f56687f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:42:46.567769 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8daf764-2110-4000-a671-38f63f56687f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:42:46.569345 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8daf764-2110-4000-a671-38f63f56687f service nova] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:42:46.579117 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 602/1204] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:42:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:42:50.484599 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6577d47e-edc3-4899-9a04-e6ca4a02db6c None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:42:50.485161 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6577d47e-edc3-4899-9a04-e6ca4a02db6c None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:42:50.485161 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6577d47e-edc3-4899-9a04-e6ca4a02db6c None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:42:50.486527 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 603/1205] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 15 08:42:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:42:50.491728 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd57561-33a0-48b9-a362-d1230b12f231 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:42:50.492118 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd57561-33a0-48b9-a362-d1230b12f231 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:42:50.492899 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd57561-33a0-48b9-a362-d1230b12f231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:42:50.512573 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-2cd57561-33a0-48b9-a362-d1230b12f231 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:42:50.513792 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-2cd57561-33a0-48b9-a362-d1230b12f231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:42:50.521431 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 603/1206] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 15 08:42:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 08:42:50.807730 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6577d47e-edc3-4899-9a04-e6ca4a02db6c None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:42:50.809492 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-6577d47e-edc3-4899-9a04-e6ca4a02db6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:42:50.885110 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6577d47e-edc3-4899-9a04-e6ca4a02db6c None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:42:50.888613 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8a1a0f-5e67-4ac7-bc1a-ed71dfb84a84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:42:50.889206 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8a1a0f-5e67-4ac7-bc1a-ed71dfb84a84 service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:42:50.889484 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8a1a0f-5e67-4ac7-bc1a-ed71dfb84a84 service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:42:50.889786 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8a1a0f-5e67-4ac7-bc1a-ed71dfb84a84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:42:50.898660 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8a1a0f-5e67-4ac7-bc1a-ed71dfb84a84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:42:50.900080 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8a1a0f-5e67-4ac7-bc1a-ed71dfb84a84 service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:42:50.909209 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 604/1207] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:42:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:42:53.035099 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68258bc4-21b0-47ec-a6c7-db18ecd727a3 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:42:53.035666 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68258bc4-21b0-47ec-a6c7-db18ecd727a3 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:42:53.035666 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68258bc4-21b0-47ec-a6c7-db18ecd727a3 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:42:53.037000 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 604/1208] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 15 08:42:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:42:53.042352 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4f8147-7272-4ef5-ac35-e5fc67e75183 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:42:53.042791 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4f8147-7272-4ef5-ac35-e5fc67e75183 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:42:53.042966 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4f8147-7272-4ef5-ac35-e5fc67e75183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:42:53.063750 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-8d4f8147-7272-4ef5-ac35-e5fc67e75183 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:42:53.064932 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-8d4f8147-7272-4ef5-ac35-e5fc67e75183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:42:53.073089 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 605/1209] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 15 08:42:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:42:53.362274 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68258bc4-21b0-47ec-a6c7-db18ecd727a3 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:42:53.372745 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68258bc4-21b0-47ec-a6c7-db18ecd727a3 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:42:53.376230 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8118316c-7230-47c7-8e36-1965cc6d413c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:42:53.376820 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8118316c-7230-47c7-8e36-1965cc6d413c service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:42:53.377103 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8118316c-7230-47c7-8e36-1965cc6d413c service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:42:53.377377 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8118316c-7230-47c7-8e36-1965cc6d413c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:42:53.386408 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8118316c-7230-47c7-8e36-1965cc6d413c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:42:53.387842 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8118316c-7230-47c7-8e36-1965cc6d413c service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:42:53.397560 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 605/1210] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:42:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:42:57.007404 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d8695-9d86-4b2f-b66d-3240f4c5e44f None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:42:57.008526 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d8695-9d86-4b2f-b66d-3240f4c5e44f None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:42:57.008622 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d8695-9d86-4b2f-b66d-3240f4c5e44f None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:42:57.009935 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 606/1211] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 15 08:42:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:42:57.015241 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219be43d-4f16-460d-9595-0c1e4af45d2d None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:42:57.015655 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219be43d-4f16-460d-9595-0c1e4af45d2d None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:42:57.015797 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219be43d-4f16-460d-9595-0c1e4af45d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:42:57.023700 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=81 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:42:57.036490 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-219be43d-4f16-460d-9595-0c1e4af45d2d None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:42:57.037711 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-219be43d-4f16-460d-9595-0c1e4af45d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:42:57.045434 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 606/1212] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 15 08:42:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:42:57.348140 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672d8695-9d86-4b2f-b66d-3240f4c5e44f None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:42:57.349909 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-672d8695-9d86-4b2f-b66d-3240f4c5e44f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:42:57.415017 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:42:57.428300 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672d8695-9d86-4b2f-b66d-3240f4c5e44f None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:42:57.432469 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a582307-13ce-4d80-b12e-8ba6e5c381f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:42:57.433067 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a582307-13ce-4d80-b12e-8ba6e5c381f4 service cinder] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:42:57.433347 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a582307-13ce-4d80-b12e-8ba6e5c381f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:42:57.433621 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a582307-13ce-4d80-b12e-8ba6e5c381f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:42:57.442413 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a582307-13ce-4d80-b12e-8ba6e5c381f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:42:57.443839 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a582307-13ce-4d80-b12e-8ba6e5c381f4 service cinder] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:42:57.453066 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 607/1213] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:42:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:42:59.562046 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc6a61-8c36-4b40-a58c-bfb1fabf654d None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:42:59.562618 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc6a61-8c36-4b40-a58c-bfb1fabf654d None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:42:59.562618 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc6a61-8c36-4b40-a58c-bfb1fabf654d None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:42:59.563946 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 607/1214] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 15 08:42:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:42:59.569276 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c5023a-f035-4efb-b1bd-1e7e6f0592c0 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:42:59.569728 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c5023a-f035-4efb-b1bd-1e7e6f0592c0 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:42:59.569876 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c5023a-f035-4efb-b1bd-1e7e6f0592c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:42:59.590179 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-27c5023a-f035-4efb-b1bd-1e7e6f0592c0 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:42:59.591338 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-27c5023a-f035-4efb-b1bd-1e7e6f0592c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:42:59.599097 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 608/1215] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 15 08:42:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:42:59.897767 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fc6a61-8c36-4b40-a58c-bfb1fabf654d None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:42:59.900289 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-20fc6a61-8c36-4b40-a58c-bfb1fabf654d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:42:59.913939 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fc6a61-8c36-4b40-a58c-bfb1fabf654d None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:42:59.918938 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fc915e-b7e4-43cd-84be-0109f71fed5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:42:59.919548 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc915e-b7e4-43cd-84be-0109f71fed5f service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:42:59.919836 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc915e-b7e4-43cd-84be-0109f71fed5f service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:42:59.920110 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc915e-b7e4-43cd-84be-0109f71fed5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:42:59.929236 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fc915e-b7e4-43cd-84be-0109f71fed5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:42:59.931274 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fc915e-b7e4-43cd-84be-0109f71fed5f service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:42:59.940560 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 608/1216] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:42:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:43:09.584660 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=11 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:43:09.936285 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=8 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:44:21.065170 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dcdabf-3275-4cd1-bcb9-6388ac571c74 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:21.066426 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dcdabf-3275-4cd1-bcb9-6388ac571c74 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:21.066922 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dcdabf-3275-4cd1-bcb9-6388ac571c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:21.092986 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-65dcdabf-3275-4cd1-bcb9-6388ac571c74 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:21.094576 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-65dcdabf-3275-4cd1-bcb9-6388ac571c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:21.105197 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 609/1217] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:44:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:21.115406 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd2f087-11a9-45a3-9fa2-c810ba1bef71 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:21.116027 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd2f087-11a9-45a3-9fa2-c810ba1bef71 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:21.116027 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd2f087-11a9-45a3-9fa2-c810ba1bef71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:21.146921 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-ccd2f087-11a9-45a3-9fa2-c810ba1bef71 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:21.148323 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-ccd2f087-11a9-45a3-9fa2-c810ba1bef71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:21.160015 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 609/1218] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:44:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 08:44:21.165755 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dcdabf-3275-4cd1-bcb9-6388ac571c74 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:21.176287 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dcdabf-3275-4cd1-bcb9-6388ac571c74 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:21.186661 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8675d5cb-2bc1-4db3-b1f7-0f2c39720691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:21.187612 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8675d5cb-2bc1-4db3-b1f7-0f2c39720691 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:21.188121 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8675d5cb-2bc1-4db3-b1f7-0f2c39720691 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:21.188650 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8675d5cb-2bc1-4db3-b1f7-0f2c39720691 admin admin] PATH_INFO: `/v3/domains` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:21.189530 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8675d5cb-2bc1-4db3-b1f7-0f2c39720691 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:44:21.190090 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8675d5cb-2bc1-4db3-b1f7-0f2c39720691 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:21.191936 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8675d5cb-2bc1-4db3-b1f7-0f2c39720691 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:21.205494 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 610/1219] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 08:44:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:44:21.213831 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd2f087-11a9-45a3-9fa2-c810ba1bef71 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:21.223399 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd2f087-11a9-45a3-9fa2-c810ba1bef71 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:21.237473 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe06575-b8c3-4074-a2e1-89dd05b526e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:21.238133 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe06575-b8c3-4074-a2e1-89dd05b526e3 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:21.238413 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe06575-b8c3-4074-a2e1-89dd05b526e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:21.238823 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe06575-b8c3-4074-a2e1-89dd05b526e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:21.239620 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe06575-b8c3-4074-a2e1-89dd05b526e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:21.241534 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe06575-b8c3-4074-a2e1-89dd05b526e3 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:21.304392 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 610/1220] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 08:44:21 2022] POST /identity/v3/projects => generated 375 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:44:21.308770 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8675d5cb-2bc1-4db3-b1f7-0f2c39720691 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:21.325114 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8675d5cb-2bc1-4db3-b1f7-0f2c39720691 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:21.413328 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de8a4e-7cec-4b36-8e22-04744a531ae3 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:21.413551 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeba48f-e0e2-4e70-a764-243ba5271238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:21.413666 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de8a4e-7cec-4b36-8e22-04744a531ae3 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:21.413962 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de8a4e-7cec-4b36-8e22-04744a531ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:21.414082 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeba48f-e0e2-4e70-a764-243ba5271238 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:21.414250 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeba48f-e0e2-4e70-a764-243ba5271238 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:21.414529 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeba48f-e0e2-4e70-a764-243ba5271238 admin admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:21.415203 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeba48f-e0e2-4e70-a764-243ba5271238 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:21.417199 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeba48f-e0e2-4e70-a764-243ba5271238 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:21.459633 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 611/1221] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:44:21 2022] POST /identity/v3/users => generated 288 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:44:21.460720 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-c8de8a4e-7cec-4b36-8e22-04744a531ae3 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:21.462024 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-c8de8a4e-7cec-4b36-8e22-04744a531ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:21.467171 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeba48f-e0e2-4e70-a764-243ba5271238 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:21.483379 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 611/1222] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:44:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:21.485650 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeba48f-e0e2-4e70-a764-243ba5271238 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:21.496115 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7319745a-edec-4024-9f3a-de6aab6610b6 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:21.496597 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7319745a-edec-4024-9f3a-de6aab6610b6 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:21.496880 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7319745a-edec-4024-9f3a-de6aab6610b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:21.497009 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72edd2a0-c889-4c20-9943-c2a0d4f83c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:21.497559 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72edd2a0-c889-4c20-9943-c2a0d4f83c01 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:21.498025 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72edd2a0-c889-4c20-9943-c2a0d4f83c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:21.498447 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72edd2a0-c889-4c20-9943-c2a0d4f83c01 admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:21.499108 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72edd2a0-c889-4c20-9943-c2a0d4f83c01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:21.500811 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72edd2a0-c889-4c20-9943-c2a0d4f83c01 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:21.518662 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 612/1223] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:44:21 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:44:21.519968 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc921552-4df7-4fb5-9cfc-57b2061b24f2 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:21.520257 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc921552-4df7-4fb5-9cfc-57b2061b24f2 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:21.522200 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc921552-4df7-4fb5-9cfc-57b2061b24f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:21.526741 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-7319745a-edec-4024-9f3a-de6aab6610b6 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:21.527959 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-7319745a-edec-4024-9f3a-de6aab6610b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:21.551498 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 612/1224] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:44:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 08:44:21.552432 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7319745a-edec-4024-9f3a-de6aab6610b6 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:21.558780 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-cc921552-4df7-4fb5-9cfc-57b2061b24f2 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:21.562964 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7319745a-edec-4024-9f3a-de6aab6610b6 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:21.563235 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-cc921552-4df7-4fb5-9cfc-57b2061b24f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:21.575056 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16239220-c89d-48e8-a3d8-bbe2f335b65b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:21.575742 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16239220-c89d-48e8-a3d8-bbe2f335b65b admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:21.575980 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16239220-c89d-48e8-a3d8-bbe2f335b65b admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:21.576245 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16239220-c89d-48e8-a3d8-bbe2f335b65b admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:21.576893 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16239220-c89d-48e8-a3d8-bbe2f335b65b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:21.578521 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16239220-c89d-48e8-a3d8-bbe2f335b65b admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:21.584099 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 613/1225] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:44:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:21.585979 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073502b-d9ae-42aa-a52e-1b7fd3aa1d11 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:21.586267 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073502b-d9ae-42aa-a52e-1b7fd3aa1d11 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:21.586544 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073502b-d9ae-42aa-a52e-1b7fd3aa1d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:21.606935 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 613/1226] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:44:21 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:44:21.608256 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90907ca-fca2-427d-94f3-a38d7979f703 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:21.612776 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90907ca-fca2-427d-94f3-a38d7979f703 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:21.613081 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90907ca-fca2-427d-94f3-a38d7979f703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:21.631158 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-f073502b-d9ae-42aa-a52e-1b7fd3aa1d11 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:21.632740 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-f073502b-d9ae-42aa-a52e-1b7fd3aa1d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:21.651816 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-c90907ca-fca2-427d-94f3-a38d7979f703 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:21.657835 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 614/1227] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:44:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:21.658754 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f073502b-d9ae-42aa-a52e-1b7fd3aa1d11 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:21.662550 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-c90907ca-fca2-427d-94f3-a38d7979f703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:21.675012 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f073502b-d9ae-42aa-a52e-1b7fd3aa1d11 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:21.676977 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 614/1228] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:44:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:21.678393 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb1452f-009a-4b82-adf7-8ff4710d2c96 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:21.678691 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb1452f-009a-4b82-adf7-8ff4710d2c96 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:21.678972 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb1452f-009a-4b82-adf7-8ff4710d2c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:21.679841 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b056d8-eaf2-490a-b6f4-156ef63c3975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:21.680448 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b056d8-eaf2-490a-b6f4-156ef63c3975 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:21.680821 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b056d8-eaf2-490a-b6f4-156ef63c3975 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:21.681623 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b056d8-eaf2-490a-b6f4-156ef63c3975 admin admin] PATH_INFO: `/v3/domains` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:21.682303 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b056d8-eaf2-490a-b6f4-156ef63c3975 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:44:21.682612 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b056d8-eaf2-490a-b6f4-156ef63c3975 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:21.684215 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b056d8-eaf2-490a-b6f4-156ef63c3975 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:21.697908 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 615/1229] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 08:44:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:44:21.698716 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b056d8-eaf2-490a-b6f4-156ef63c3975 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:21.700555 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-3eb1452f-009a-4b82-adf7-8ff4710d2c96 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:21.701788 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-3eb1452f-009a-4b82-adf7-8ff4710d2c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:21.708683 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b056d8-eaf2-490a-b6f4-156ef63c3975 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:21.709864 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 615/1230] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:44:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:21.711040 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede9b3f-5861-419f-9794-04847fe7ecd8 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:21.711326 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede9b3f-5861-419f-9794-04847fe7ecd8 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:21.711602 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede9b3f-5861-419f-9794-04847fe7ecd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:21.716795 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d463398c-bb1e-4352-a80c-4c0d77210010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:21.717447 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d463398c-bb1e-4352-a80c-4c0d77210010 admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:21.717756 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d463398c-bb1e-4352-a80c-4c0d77210010 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:21.718037 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d463398c-bb1e-4352-a80c-4c0d77210010 admin admin] PATH_INFO: `/v3/projects/becb9cd46453447a87846cf26561a190/users/a5d85f7efca040cc81253dfbf0de7c42/roles/3248f0268738489b883041851d681bb7` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:21.731001 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-5ede9b3f-5861-419f-9794-04847fe7ecd8 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:21.732208 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-5ede9b3f-5861-419f-9794-04847fe7ecd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:21.733004 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d463398c-bb1e-4352-a80c-4c0d77210010 admin admin] RBAC: Authorizing `identity:create_grant(project_id=becb9cd46453447a87846cf26561a190, user_id=a5d85f7efca040cc81253dfbf0de7c42, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:21.735689 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d463398c-bb1e-4352-a80c-4c0d77210010 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:21.740067 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 616/1231] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:44:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:21.741432 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbcf140-df6d-4715-a158-9766704a5969 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:21.741820 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbcf140-df6d-4715-a158-9766704a5969 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:21.742116 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbcf140-df6d-4715-a158-9766704a5969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:21.752155 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 616/1232] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:44:21 2022] PUT /identity/v3/projects/becb9cd46453447a87846cf26561a190/users/a5d85f7efca040cc81253dfbf0de7c42/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:44:21.753822 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa8a09-2d78-4ea8-9cb2-5de04a82113c None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:21.754113 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa8a09-2d78-4ea8-9cb2-5de04a82113c None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:21.754382 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa8a09-2d78-4ea8-9cb2-5de04a82113c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:21.763579 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-4bbcf140-df6d-4715-a158-9766704a5969 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:21.764978 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-4bbcf140-df6d-4715-a158-9766704a5969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:21.774867 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-edaa8a09-2d78-4ea8-9cb2-5de04a82113c None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:21.776044 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-edaa8a09-2d78-4ea8-9cb2-5de04a82113c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:21.825907 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 617/1233] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:44:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:21.827074 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b347cdd-3f4d-46cd-92f5-6ee1838f8dbc None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:21.827358 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b347cdd-3f4d-46cd-92f5-6ee1838f8dbc None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:21.827636 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b347cdd-3f4d-46cd-92f5-6ee1838f8dbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:21.846768 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-6b347cdd-3f4d-46cd-92f5-6ee1838f8dbc None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:21.847124 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 617/1234] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:44:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:21.848055 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-6b347cdd-3f4d-46cd-92f5-6ee1838f8dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:21.848509 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de327bb8-12c9-4501-a11b-de94070370e5 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:21.848801 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de327bb8-12c9-4501-a11b-de94070370e5 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:21.849107 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de327bb8-12c9-4501-a11b-de94070370e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:21.857924 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 618/1235] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:44:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:21.858709 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b347cdd-3f4d-46cd-92f5-6ee1838f8dbc None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:21.867882 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b347cdd-3f4d-46cd-92f5-6ee1838f8dbc None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:21.873184 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5909c4dc-2651-4b2f-bced-5c29af68c824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:21.873819 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5909c4dc-2651-4b2f-bced-5c29af68c824 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:21.873975 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-de327bb8-12c9-4501-a11b-de94070370e5 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:21.874091 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5909c4dc-2651-4b2f-bced-5c29af68c824 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:21.874368 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5909c4dc-2651-4b2f-bced-5c29af68c824 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:21.874995 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5909c4dc-2651-4b2f-bced-5c29af68c824 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:21.875116 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-de327bb8-12c9-4501-a11b-de94070370e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:21.876959 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5909c4dc-2651-4b2f-bced-5c29af68c824 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:21.882990 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 618/1236] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:44:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:21.883861 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de327bb8-12c9-4501-a11b-de94070370e5 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:21.888571 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:44:21.895203 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de327bb8-12c9-4501-a11b-de94070370e5 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:21.901752 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71192aea-ebfc-4c4f-94ad-5a7c3a3d2b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:21.902352 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71192aea-ebfc-4c4f-94ad-5a7c3a3d2b0b admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:21.902633 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71192aea-ebfc-4c4f-94ad-5a7c3a3d2b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:21.902908 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71192aea-ebfc-4c4f-94ad-5a7c3a3d2b0b admin admin] PATH_INFO: `/v3/domains` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:21.903552 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71192aea-ebfc-4c4f-94ad-5a7c3a3d2b0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:44:21.903890 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71192aea-ebfc-4c4f-94ad-5a7c3a3d2b0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:21.905536 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71192aea-ebfc-4c4f-94ad-5a7c3a3d2b0b admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:21.905652 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 619/1237] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 08:44:21 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:44:21.907135 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ba884c-e07c-45d6-ac13-09ac74228d24 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:21.907525 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ba884c-e07c-45d6-ac13-09ac74228d24 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:21.907906 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ba884c-e07c-45d6-ac13-09ac74228d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:21.925462 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 619/1238] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 08:44:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:44:21.925462 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f580a-eb1f-41c5-90f1-eb898e48f3a8 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:21.925462 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f580a-eb1f-41c5-90f1-eb898e48f3a8 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:21.925462 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f580a-eb1f-41c5-90f1-eb898e48f3a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:21.927738 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-b8ba884c-e07c-45d6-ac13-09ac74228d24 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:21.928950 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-b8ba884c-e07c-45d6-ac13-09ac74228d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:21.942062 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-8b2f580a-eb1f-41c5-90f1-eb898e48f3a8 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:21.943265 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-8b2f580a-eb1f-41c5-90f1-eb898e48f3a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:21.975074 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:44:21.991269 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 620/1239] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:44:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:21.992512 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb3e3ae-f22f-4ee4-b345-23f4ff4998ad None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:21.992793 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb3e3ae-f22f-4ee4-b345-23f4ff4998ad None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:21.993065 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb3e3ae-f22f-4ee4-b345-23f4ff4998ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:22.006267 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 620/1240] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:44:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:22.007822 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59c6591-8855-4faa-a26e-5626718975b3 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:22.008113 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59c6591-8855-4faa-a26e-5626718975b3 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:22.008395 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59c6591-8855-4faa-a26e-5626718975b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:22.015745 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-dfb3e3ae-f22f-4ee4-b345-23f4ff4998ad None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:22.017147 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-dfb3e3ae-f22f-4ee4-b345-23f4ff4998ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:22.025825 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 621/1241] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:44:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:22.026611 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb3e3ae-f22f-4ee4-b345-23f4ff4998ad None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:22.028856 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-c59c6591-8855-4faa-a26e-5626718975b3 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:22.030440 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-c59c6591-8855-4faa-a26e-5626718975b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:22.035793 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb3e3ae-f22f-4ee4-b345-23f4ff4998ad None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:22.038848 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 621/1242] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:44:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:22.039658 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59c6591-8855-4faa-a26e-5626718975b3 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:22.044763 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5ec549-f6dd-46d1-b966-c6b8fae1fbfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:22.045365 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5ec549-f6dd-46d1-b966-c6b8fae1fbfa admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:22.045643 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5ec549-f6dd-46d1-b966-c6b8fae1fbfa admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:22.045945 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5ec549-f6dd-46d1-b966-c6b8fae1fbfa admin admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:22.046588 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5ec549-f6dd-46d1-b966-c6b8fae1fbfa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:44:22.046885 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5ec549-f6dd-46d1-b966-c6b8fae1fbfa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:22.047882 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: WARNING py.warnings [None req-4b5ec549-f6dd-46d1-b966-c6b8fae1fbfa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 08:44:22.047882 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: warnings.warn(msg) Oct 15 08:44:22.047882 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: Oct 15 08:44:22.048759 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5ec549-f6dd-46d1-b966-c6b8fae1fbfa admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:22.049792 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59c6591-8855-4faa-a26e-5626718975b3 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:22.054397 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48921181-0c62-4f3e-88f1-c7a71d610ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:22.054984 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48921181-0c62-4f3e-88f1-c7a71d610ff1 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:22.055263 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48921181-0c62-4f3e-88f1-c7a71d610ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:22.055538 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48921181-0c62-4f3e-88f1-c7a71d610ff1 admin admin] PATH_INFO: `/v3/domains` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:22.056206 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48921181-0c62-4f3e-88f1-c7a71d610ff1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:44:22.056532 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48921181-0c62-4f3e-88f1-c7a71d610ff1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:22.058148 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48921181-0c62-4f3e-88f1-c7a71d610ff1 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:22.060757 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 622/1243] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 08:44:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:44:22.061704 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5ec549-f6dd-46d1-b966-c6b8fae1fbfa admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:22.071464 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5ec549-f6dd-46d1-b966-c6b8fae1fbfa admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:22.074198 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 622/1244] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 08:44:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:44:22.075013 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48921181-0c62-4f3e-88f1-c7a71d610ff1 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:22.082047 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fc7005-5ff8-4ab1-a17b-ddf86fc20ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:22.082759 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc7005-5ff8-4ab1-a17b-ddf86fc20ac5 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:22.083045 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc7005-5ff8-4ab1-a17b-ddf86fc20ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:22.083347 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc7005-5ff8-4ab1-a17b-ddf86fc20ac5 admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:22.083976 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fc7005-5ff8-4ab1-a17b-ddf86fc20ac5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:22.085005 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: WARNING py.warnings [None req-a5fc7005-5ff8-4ab1-a17b-ddf86fc20ac5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 08:44:22.085005 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: warnings.warn(msg) Oct 15 08:44:22.085005 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: Oct 15 08:44:22.086300 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fc7005-5ff8-4ab1-a17b-ddf86fc20ac5 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:22.089610 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48921181-0c62-4f3e-88f1-c7a71d610ff1 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:22.094108 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af389e3e-aaf6-40ad-848e-775f3c604cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:22.094691 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af389e3e-aaf6-40ad-848e-775f3c604cf2 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:22.094968 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af389e3e-aaf6-40ad-848e-775f3c604cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:22.095238 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af389e3e-aaf6-40ad-848e-775f3c604cf2 admin admin] PATH_INFO: `/v3/projects` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:22.095847 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af389e3e-aaf6-40ad-848e-775f3c604cf2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:22.096855 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: WARNING py.warnings [None req-af389e3e-aaf6-40ad-848e-775f3c604cf2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 08:44:22.096855 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: warnings.warn(msg) Oct 15 08:44:22.096855 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: Oct 15 08:44:22.098101 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af389e3e-aaf6-40ad-848e-775f3c604cf2 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:22.111220 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 623/1245] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:44:22 2022] POST /identity/v3/users => generated 289 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:44:22.112272 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fc7005-5ff8-4ab1-a17b-ddf86fc20ac5 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:22.124602 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fc7005-5ff8-4ab1-a17b-ddf86fc20ac5 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:22.127169 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 623/1246] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 08:44:22 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:44:22.128012 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af389e3e-aaf6-40ad-848e-775f3c604cf2 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:22.129369 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7a592c-59ad-413b-bf04-3e469b8bbf56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:22.130221 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7a592c-59ad-413b-bf04-3e469b8bbf56 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:22.130538 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7a592c-59ad-413b-bf04-3e469b8bbf56 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:22.130816 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7a592c-59ad-413b-bf04-3e469b8bbf56 admin admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:22.131464 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7a592c-59ad-413b-bf04-3e469b8bbf56 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:44:22.131775 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7a592c-59ad-413b-bf04-3e469b8bbf56 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:22.133597 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7a592c-59ad-413b-bf04-3e469b8bbf56 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:22.145585 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 624/1247] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 08:44:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:44:22.146812 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7a592c-59ad-413b-bf04-3e469b8bbf56 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:22.157247 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7a592c-59ad-413b-bf04-3e469b8bbf56 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:22.182760 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af389e3e-aaf6-40ad-848e-775f3c604cf2 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:22.186421 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4922f00b-dbde-4605-ac1d-f787b3b2a379 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:22.187030 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4922f00b-dbde-4605-ac1d-f787b3b2a379 service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:22.187316 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4922f00b-dbde-4605-ac1d-f787b3b2a379 service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:22.187595 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4922f00b-dbde-4605-ac1d-f787b3b2a379 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:22.215068 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1381eb7a-16bb-4e2c-abd0-d6e216e913f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:22.215674 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1381eb7a-16bb-4e2c-abd0-d6e216e913f3 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:22.215956 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1381eb7a-16bb-4e2c-abd0-d6e216e913f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:22.216235 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1381eb7a-16bb-4e2c-abd0-d6e216e913f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:22.216874 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1381eb7a-16bb-4e2c-abd0-d6e216e913f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:44:22.217180 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1381eb7a-16bb-4e2c-abd0-d6e216e913f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:22.218809 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1381eb7a-16bb-4e2c-abd0-d6e216e913f3 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:22.231438 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 625/1248] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 08:44:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:44:22.233103 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1381eb7a-16bb-4e2c-abd0-d6e216e913f3 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:22.243195 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1381eb7a-16bb-4e2c-abd0-d6e216e913f3 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:22.247676 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4922f00b-dbde-4605-ac1d-f787b3b2a379 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:22.247875 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84029762-7700-4804-a3ad-50bae440e6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:22.248464 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84029762-7700-4804-a3ad-50bae440e6af admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:22.248752 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84029762-7700-4804-a3ad-50bae440e6af admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:22.249037 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84029762-7700-4804-a3ad-50bae440e6af admin admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:22.249157 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4922f00b-dbde-4605-ac1d-f787b3b2a379 service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:22.249672 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84029762-7700-4804-a3ad-50bae440e6af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:44:22.250005 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84029762-7700-4804-a3ad-50bae440e6af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:22.251574 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84029762-7700-4804-a3ad-50bae440e6af admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:22.252930 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:44:22.259130 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 624/1249] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:44:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:44:22.263428 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4922f00b-dbde-4605-ac1d-f787b3b2a379 service neutron] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:22.273008 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 626/1250] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 08:44:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:44:22.273008 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84029762-7700-4804-a3ad-50bae440e6af admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:22.275641 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4922f00b-dbde-4605-ac1d-f787b3b2a379 service neutron] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:22.280756 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d11024c-68e9-4c24-aa89-3e0b6b896d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:22.281257 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11024c-68e9-4c24-aa89-3e0b6b896d37 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:22.281536 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11024c-68e9-4c24-aa89-3e0b6b896d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:22.281839 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11024c-68e9-4c24-aa89-3e0b6b896d37 admin admin] PATH_INFO: `/v3/projects` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:22.282471 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d11024c-68e9-4c24-aa89-3e0b6b896d37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:22.283930 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84029762-7700-4804-a3ad-50bae440e6af admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:22.284648 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d11024c-68e9-4c24-aa89-3e0b6b896d37 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:22.288971 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f325e9-81df-4e1c-9545-08513475e72d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:22.289614 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f325e9-81df-4e1c-9545-08513475e72d admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:22.290100 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f325e9-81df-4e1c-9545-08513475e72d admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:22.290928 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f325e9-81df-4e1c-9545-08513475e72d admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:22.291582 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f325e9-81df-4e1c-9545-08513475e72d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:22.293512 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f325e9-81df-4e1c-9545-08513475e72d admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:22.316329 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 625/1251] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 08:44:22 2022] POST /identity/v3/projects => generated 405 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:44:22.317320 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d11024c-68e9-4c24-aa89-3e0b6b896d37 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:22.330596 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 627/1252] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 08:44:22 2022] POST /identity/v3/projects => generated 389 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:44:22.330596 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f325e9-81df-4e1c-9545-08513475e72d admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:22.398730 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d11024c-68e9-4c24-aa89-3e0b6b896d37 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:22.422736 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f325e9-81df-4e1c-9545-08513475e72d admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:22.428180 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950eb263-34c5-47fc-8877-243bf25c2e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:22.428811 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950eb263-34c5-47fc-8877-243bf25c2e4b admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:22.429110 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950eb263-34c5-47fc-8877-243bf25c2e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:22.429385 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950eb263-34c5-47fc-8877-243bf25c2e4b admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:22.430466 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950eb263-34c5-47fc-8877-243bf25c2e4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:22.433433 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950eb263-34c5-47fc-8877-243bf25c2e4b admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:22.443810 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:44:22.465639 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 628/1253] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:44:22 2022] POST /identity/v3/users => generated 292 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:44:22.467222 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950eb263-34c5-47fc-8877-243bf25c2e4b admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:22.477829 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950eb263-34c5-47fc-8877-243bf25c2e4b admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:22.482480 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e2dab6-5697-41d1-8d13-e6b90a1d1442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:22.483093 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2dab6-5697-41d1-8d13-e6b90a1d1442 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:22.483370 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2dab6-5697-41d1-8d13-e6b90a1d1442 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:22.483641 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2dab6-5697-41d1-8d13-e6b90a1d1442 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:22.484272 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e2dab6-5697-41d1-8d13-e6b90a1d1442 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:22.486291 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e2dab6-5697-41d1-8d13-e6b90a1d1442 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:22.490825 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281093f6-9494-4d16-b617-166bec043231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:22.491703 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281093f6-9494-4d16-b617-166bec043231 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:22.492052 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281093f6-9494-4d16-b617-166bec043231 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:22.492549 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281093f6-9494-4d16-b617-166bec043231 admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:22.493289 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281093f6-9494-4d16-b617-166bec043231 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:22.495339 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281093f6-9494-4d16-b617-166bec043231 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:22.509997 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 626/1254] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:44:22 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:44:22.510964 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281093f6-9494-4d16-b617-166bec043231 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:22.522021 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 629/1255] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 08:44:22 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:44:22.522021 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e2dab6-5697-41d1-8d13-e6b90a1d1442 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:22.525592 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281093f6-9494-4d16-b617-166bec043231 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:22.592830 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e2dab6-5697-41d1-8d13-e6b90a1d1442 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:22.597365 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497336be-0f1d-416e-8043-28fe64f856a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:22.598171 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497336be-0f1d-416e-8043-28fe64f856a0 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:22.598460 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497336be-0f1d-416e-8043-28fe64f856a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:22.598738 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497336be-0f1d-416e-8043-28fe64f856a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:22.599254 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5674a6dd-1e6a-4a12-86ee-65463bbe565f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:22.599579 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497336be-0f1d-416e-8043-28fe64f856a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:22.600199 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5674a6dd-1e6a-4a12-86ee-65463bbe565f admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:22.600585 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5674a6dd-1e6a-4a12-86ee-65463bbe565f admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:22.600962 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5674a6dd-1e6a-4a12-86ee-65463bbe565f admin admin] PATH_INFO: `/v3/projects` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:22.601510 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497336be-0f1d-416e-8043-28fe64f856a0 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:22.603207 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5674a6dd-1e6a-4a12-86ee-65463bbe565f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:22.606005 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5674a6dd-1e6a-4a12-86ee-65463bbe565f admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:22.635155 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 630/1256] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 08:44:22 2022] POST /identity/v3/projects => generated 391 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:44:22.636266 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497336be-0f1d-416e-8043-28fe64f856a0 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:22.652897 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 627/1257] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 08:44:22 2022] POST /identity/v3/projects => generated 367 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:44:22.654400 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5674a6dd-1e6a-4a12-86ee-65463bbe565f admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:22.661954 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:44:22.712094 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497336be-0f1d-416e-8043-28fe64f856a0 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:22.734646 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5674a6dd-1e6a-4a12-86ee-65463bbe565f admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:22.739903 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b889bef-409b-42b1-9ff8-5669a4ff765b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:22.740520 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b889bef-409b-42b1-9ff8-5669a4ff765b admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:22.740802 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b889bef-409b-42b1-9ff8-5669a4ff765b admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:22.741078 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b889bef-409b-42b1-9ff8-5669a4ff765b admin admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:22.741741 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b889bef-409b-42b1-9ff8-5669a4ff765b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:22.744318 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b889bef-409b-42b1-9ff8-5669a4ff765b admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:22.772626 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 628/1258] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:44:22 2022] POST /identity/v3/users => generated 295 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:44:22.772626 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b889bef-409b-42b1-9ff8-5669a4ff765b admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:22.779165 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0802d13f-a1a7-4f82-9ee2-acf5538dc42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:22.779896 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802d13f-a1a7-4f82-9ee2-acf5538dc42e admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:22.780184 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802d13f-a1a7-4f82-9ee2-acf5538dc42e admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:22.780517 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802d13f-a1a7-4f82-9ee2-acf5538dc42e admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:22.781162 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0802d13f-a1a7-4f82-9ee2-acf5538dc42e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:22.783271 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b889bef-409b-42b1-9ff8-5669a4ff765b admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:22.783405 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0802d13f-a1a7-4f82-9ee2-acf5538dc42e admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:22.787830 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb621c6-130e-4077-9b65-99458ed1315d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:22.788465 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb621c6-130e-4077-9b65-99458ed1315d admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:22.788750 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb621c6-130e-4077-9b65-99458ed1315d admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:22.789032 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb621c6-130e-4077-9b65-99458ed1315d admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:22.789703 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb621c6-130e-4077-9b65-99458ed1315d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:22.791341 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb621c6-130e-4077-9b65-99458ed1315d admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:22.792384 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:44:22.803728 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 629/1259] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:44:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:44:22.804904 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb621c6-130e-4077-9b65-99458ed1315d admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:22.809857 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 631/1260] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:44:22 2022] POST /identity/v3/users => generated 303 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:44:22.813663 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0802d13f-a1a7-4f82-9ee2-acf5538dc42e admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:22.817245 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb621c6-130e-4077-9b65-99458ed1315d admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:22.822663 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee2a3e-0c2e-4e1c-9c01-386331543fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:22.823379 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee2a3e-0c2e-4e1c-9c01-386331543fd3 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:22.823755 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee2a3e-0c2e-4e1c-9c01-386331543fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:22.824121 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee2a3e-0c2e-4e1c-9c01-386331543fd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:22.824234 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0802d13f-a1a7-4f82-9ee2-acf5538dc42e admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:22.824933 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fee2a3e-0c2e-4e1c-9c01-386331543fd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:22.826684 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fee2a3e-0c2e-4e1c-9c01-386331543fd3 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:22.829817 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e99d659-d595-4b20-815a-732d37a23fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:22.830659 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e99d659-d595-4b20-815a-732d37a23fd8 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:22.831574 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e99d659-d595-4b20-815a-732d37a23fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:22.831574 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e99d659-d595-4b20-815a-732d37a23fd8 admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:22.831912 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e99d659-d595-4b20-815a-732d37a23fd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:22.834517 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e99d659-d595-4b20-815a-732d37a23fd8 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:22.841795 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 630/1261] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:44:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:44:22.841795 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee2a3e-0c2e-4e1c-9c01-386331543fd3 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:22.852646 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee2a3e-0c2e-4e1c-9c01-386331543fd3 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:22.856890 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 632/1262] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:44:22 2022] POST /identity/v3/users => generated 292 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:44:22.857793 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d216fc9-cbfa-4deb-8022-9cd13dee5788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:22.858503 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d216fc9-cbfa-4deb-8022-9cd13dee5788 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:22.858889 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d216fc9-cbfa-4deb-8022-9cd13dee5788 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:22.859248 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d216fc9-cbfa-4deb-8022-9cd13dee5788 admin admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:22.860005 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d216fc9-cbfa-4deb-8022-9cd13dee5788 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:22.861150 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e99d659-d595-4b20-815a-732d37a23fd8 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:22.862190 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d216fc9-cbfa-4deb-8022-9cd13dee5788 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:22.871658 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e99d659-d595-4b20-815a-732d37a23fd8 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:22.877972 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f853065-8181-4b1c-a5c4-c7459bd513bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:22.878625 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f853065-8181-4b1c-a5c4-c7459bd513bb admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:22.878958 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f853065-8181-4b1c-a5c4-c7459bd513bb admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:22.879288 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f853065-8181-4b1c-a5c4-c7459bd513bb admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:22.880005 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f853065-8181-4b1c-a5c4-c7459bd513bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:22.881970 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f853065-8181-4b1c-a5c4-c7459bd513bb admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:22.888956 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 631/1263] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:44:22 2022] POST /identity/v3/users => generated 296 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:44:22.890544 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d216fc9-cbfa-4deb-8022-9cd13dee5788 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:22.907395 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d216fc9-cbfa-4deb-8022-9cd13dee5788 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:22.908131 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 633/1264] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:44:22 2022] POST /identity/v3/users => generated 284 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:44:22.909614 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f853065-8181-4b1c-a5c4-c7459bd513bb admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:22.913168 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d630560b-dabf-4374-b53b-a9834f86e0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:22.913950 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d630560b-dabf-4374-b53b-a9834f86e0e8 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:22.914281 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d630560b-dabf-4374-b53b-a9834f86e0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:22.914541 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d630560b-dabf-4374-b53b-a9834f86e0e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:22.915467 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d630560b-dabf-4374-b53b-a9834f86e0e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:22.917402 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d630560b-dabf-4374-b53b-a9834f86e0e8 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:22.922754 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f853065-8181-4b1c-a5c4-c7459bd513bb admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:22.927844 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea3533a-0b59-46d1-a264-f97d7263d344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:22.928473 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3533a-0b59-46d1-a264-f97d7263d344 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:22.928746 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3533a-0b59-46d1-a264-f97d7263d344 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:22.932585 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3533a-0b59-46d1-a264-f97d7263d344 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:22.932719 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 632/1265] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:44:22 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:44:22.932821 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea3533a-0b59-46d1-a264-f97d7263d344 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:22.932908 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d630560b-dabf-4374-b53b-a9834f86e0e8 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:22.933001 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea3533a-0b59-46d1-a264-f97d7263d344 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:22.941375 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d630560b-dabf-4374-b53b-a9834f86e0e8 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:22.947197 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435e06f4-4f64-40fb-8f23-56661a49353a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:22.947428 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 634/1266] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:44:22 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:44:22.948250 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e06f4-4f64-40fb-8f23-56661a49353a admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:22.948378 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea3533a-0b59-46d1-a264-f97d7263d344 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:22.948941 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e06f4-4f64-40fb-8f23-56661a49353a admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:22.949431 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e06f4-4f64-40fb-8f23-56661a49353a admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:22.950406 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435e06f4-4f64-40fb-8f23-56661a49353a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:22.952247 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435e06f4-4f64-40fb-8f23-56661a49353a admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:22.959636 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea3533a-0b59-46d1-a264-f97d7263d344 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:22.965390 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b278cd4-e472-40e6-900c-9931a59d09f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:22.965731 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b278cd4-e472-40e6-900c-9931a59d09f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:22.966017 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b278cd4-e472-40e6-900c-9931a59d09f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:22.966285 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b278cd4-e472-40e6-900c-9931a59d09f9 admin admin] PATH_INFO: `/v3/projects/bccc9c2f8b8f4bc3a4dbc121a32d82ad/users/2b1999640b3c41beafefe80d40bf965c/roles/3248f0268738489b883041851d681bb7` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:22.966971 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 633/1267] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:44:22 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:44:22.967742 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435e06f4-4f64-40fb-8f23-56661a49353a admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:22.977833 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435e06f4-4f64-40fb-8f23-56661a49353a admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:22.982576 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f24aea2-137f-42ce-81d4-7d5667f1feb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:22.983274 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f24aea2-137f-42ce-81d4-7d5667f1feb9 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:22.983654 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f24aea2-137f-42ce-81d4-7d5667f1feb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:22.984017 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f24aea2-137f-42ce-81d4-7d5667f1feb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:22.984724 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f24aea2-137f-42ce-81d4-7d5667f1feb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:22.985037 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b278cd4-e472-40e6-900c-9931a59d09f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bccc9c2f8b8f4bc3a4dbc121a32d82ad, user_id=2b1999640b3c41beafefe80d40bf965c, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:22.986500 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f24aea2-137f-42ce-81d4-7d5667f1feb9 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:22.987797 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b278cd4-e472-40e6-900c-9931a59d09f9 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:22.999393 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 634/1268] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:44:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:44:23.000289 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f24aea2-137f-42ce-81d4-7d5667f1feb9 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:23.007017 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 635/1269] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:44:22 2022] PUT /identity/v3/projects/bccc9c2f8b8f4bc3a4dbc121a32d82ad/users/2b1999640b3c41beafefe80d40bf965c/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:44:23.007017 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b278cd4-e472-40e6-900c-9931a59d09f9 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:23.012811 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f24aea2-137f-42ce-81d4-7d5667f1feb9 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:23.079541 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b278cd4-e472-40e6-900c-9931a59d09f9 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:23.083303 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b573bd58-7fbc-4168-a1ef-4bb7139add96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:23.083932 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b573bd58-7fbc-4168-a1ef-4bb7139add96 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:23.084208 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b573bd58-7fbc-4168-a1ef-4bb7139add96 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:23.084564 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b573bd58-7fbc-4168-a1ef-4bb7139add96 admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:23.085238 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b573bd58-7fbc-4168-a1ef-4bb7139add96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:23.087118 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b573bd58-7fbc-4168-a1ef-4bb7139add96 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:23.087269 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f3305f-87ee-4de9-b27e-6b046f40998e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:23.088393 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3305f-87ee-4de9-b27e-6b046f40998e admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:23.088534 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3305f-87ee-4de9-b27e-6b046f40998e admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:23.089033 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3305f-87ee-4de9-b27e-6b046f40998e admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:23.089798 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f3305f-87ee-4de9-b27e-6b046f40998e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:23.091521 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f3305f-87ee-4de9-b27e-6b046f40998e admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:23.099983 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 635/1270] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:44:22 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:44:23.101220 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b573bd58-7fbc-4168-a1ef-4bb7139add96 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:23.105714 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 636/1271] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:44:23 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:44:23.106681 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f3305f-87ee-4de9-b27e-6b046f40998e admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:23.113111 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b573bd58-7fbc-4168-a1ef-4bb7139add96 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:23.119008 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1f264e-6d95-41f2-8f56-77160ca76011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:23.119873 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f264e-6d95-41f2-8f56-77160ca76011 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:23.120203 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f264e-6d95-41f2-8f56-77160ca76011 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:23.120481 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f264e-6d95-41f2-8f56-77160ca76011 admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:23.121127 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1f264e-6d95-41f2-8f56-77160ca76011 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:23.121589 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f3305f-87ee-4de9-b27e-6b046f40998e admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:23.123014 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1f264e-6d95-41f2-8f56-77160ca76011 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:23.127662 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfa1f29-28c1-46ad-b643-9057fdcf0701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:23.128334 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa1f29-28c1-46ad-b643-9057fdcf0701 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:23.128617 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa1f29-28c1-46ad-b643-9057fdcf0701 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:23.128889 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa1f29-28c1-46ad-b643-9057fdcf0701 admin admin] PATH_INFO: `/v3/projects/d913701e44d244b9b5c2cda61d85cd32/users/cdfa309c37d943b992f6ca1c647557af/roles/3248f0268738489b883041851d681bb7` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:23.135714 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 636/1272] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:44:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:44:23.136847 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1f264e-6d95-41f2-8f56-77160ca76011 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:23.150543 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfa1f29-28c1-46ad-b643-9057fdcf0701 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d913701e44d244b9b5c2cda61d85cd32, user_id=cdfa309c37d943b992f6ca1c647557af, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:23.153573 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1f264e-6d95-41f2-8f56-77160ca76011 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:23.154620 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfa1f29-28c1-46ad-b643-9057fdcf0701 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:23.158472 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678218c9-1da4-47c9-9e90-f2e95d63b47f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:23.159332 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678218c9-1da4-47c9-9e90-f2e95d63b47f admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:23.159663 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678218c9-1da4-47c9-9e90-f2e95d63b47f admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:23.160031 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678218c9-1da4-47c9-9e90-f2e95d63b47f admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:23.160874 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678218c9-1da4-47c9-9e90-f2e95d63b47f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:23.162494 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678218c9-1da4-47c9-9e90-f2e95d63b47f admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:23.165877 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:44:23.176377 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 637/1274] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:44:23 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:44:23.177576 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678218c9-1da4-47c9-9e90-f2e95d63b47f admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:23.185136 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 637/1274] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:44:23 2022] PUT /identity/v3/projects/d913701e44d244b9b5c2cda61d85cd32/users/cdfa309c37d943b992f6ca1c647557af/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:44:23.186845 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d573438-cb31-4c40-813a-d3839768a75b None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:23.188347 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678218c9-1da4-47c9-9e90-f2e95d63b47f admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:23.189974 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d573438-cb31-4c40-813a-d3839768a75b None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:23.190312 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d573438-cb31-4c40-813a-d3839768a75b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:23.220560 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-1d573438-cb31-4c40-813a-d3839768a75b None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:23.222335 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-1d573438-cb31-4c40-813a-d3839768a75b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:23.264784 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ecf3d5-179a-4baa-8b07-7f9ad5e74e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:23.265778 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ecf3d5-179a-4baa-8b07-7f9ad5e74e21 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:23.266963 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ecf3d5-179a-4baa-8b07-7f9ad5e74e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:23.267409 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ecf3d5-179a-4baa-8b07-7f9ad5e74e21 admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:23.268179 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ecf3d5-179a-4baa-8b07-7f9ad5e74e21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:23.270069 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ecf3d5-179a-4baa-8b07-7f9ad5e74e21 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:23.282117 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 638/1275] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:44:23 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:44:23.282623 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:INSERT=11,keystone:SELECT=89 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:44:23.283602 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ecf3d5-179a-4baa-8b07-7f9ad5e74e21 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:23.296386 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ecf3d5-179a-4baa-8b07-7f9ad5e74e21 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:23.301616 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dcf2a3-f567-4fae-8bed-8ec8ef121dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:23.302507 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dcf2a3-f567-4fae-8bed-8ec8ef121dac admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:23.302899 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dcf2a3-f567-4fae-8bed-8ec8ef121dac admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:23.303268 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dcf2a3-f567-4fae-8bed-8ec8ef121dac admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:23.304007 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dcf2a3-f567-4fae-8bed-8ec8ef121dac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:23.305600 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 638/1276] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:44:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:23.306944 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dcf2a3-f567-4fae-8bed-8ec8ef121dac admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:23.307189 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d573438-cb31-4c40-813a-d3839768a75b None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:23.323696 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d573438-cb31-4c40-813a-d3839768a75b None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:23.329039 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f900b643-4318-4bb1-9fe4-0259290171bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:23.329222 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 639/1277] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:44:23 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:44:23.329520 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f900b643-4318-4bb1-9fe4-0259290171bf admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:23.329966 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f900b643-4318-4bb1-9fe4-0259290171bf admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:23.330242 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f900b643-4318-4bb1-9fe4-0259290171bf admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:23.330472 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dcf2a3-f567-4fae-8bed-8ec8ef121dac admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:23.331002 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f900b643-4318-4bb1-9fe4-0259290171bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:23.332682 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f900b643-4318-4bb1-9fe4-0259290171bf admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:23.341251 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dcf2a3-f567-4fae-8bed-8ec8ef121dac admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:23.345076 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 639/1278] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:44:23 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:44:23.345996 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f900b643-4318-4bb1-9fe4-0259290171bf admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:23.349815 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016ac13b-0e97-4196-bebf-b3230edcf30b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:23.350619 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ac13b-0e97-4196-bebf-b3230edcf30b admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:23.351001 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ac13b-0e97-4196-bebf-b3230edcf30b admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:23.351373 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ac13b-0e97-4196-bebf-b3230edcf30b admin admin] PATH_INFO: `/v3/projects/23e3a95b1487450cad043a820d321f26/users/388ca829d87348bd9b524fd9a80e578b/roles/3248f0268738489b883041851d681bb7` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:23.359844 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f900b643-4318-4bb1-9fe4-0259290171bf admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:23.365902 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd22b353-42b0-4a04-8755-b94889aff3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:23.366928 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd22b353-42b0-4a04-8755-b94889aff3e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:23.367371 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd22b353-42b0-4a04-8755-b94889aff3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:23.367774 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd22b353-42b0-4a04-8755-b94889aff3e9 admin admin] PATH_INFO: `/v3/projects/ea1cbf684f2b4fc2b5329610334d4762/users/e930e2400bac47c481332a72857166bb/roles/3248f0268738489b883041851d681bb7` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:23.370267 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016ac13b-0e97-4196-bebf-b3230edcf30b admin admin] RBAC: Authorizing `identity:create_grant(project_id=23e3a95b1487450cad043a820d321f26, user_id=388ca829d87348bd9b524fd9a80e578b, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:23.373742 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016ac13b-0e97-4196-bebf-b3230edcf30b admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:23.391395 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd22b353-42b0-4a04-8755-b94889aff3e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea1cbf684f2b4fc2b5329610334d4762, user_id=e930e2400bac47c481332a72857166bb, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:23.394775 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd22b353-42b0-4a04-8755-b94889aff3e9 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:23.403472 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 640/1279] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:44:23 2022] PUT /identity/v3/projects/23e3a95b1487450cad043a820d321f26/users/388ca829d87348bd9b524fd9a80e578b/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:44:23.408365 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dcb109-3781-42b8-a2dc-aef6301e2b9f None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:23.409768 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dcb109-3781-42b8-a2dc-aef6301e2b9f None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:23.410601 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dcb109-3781-42b8-a2dc-aef6301e2b9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:23.414998 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 640/1280] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:44:23 2022] PUT /identity/v3/projects/ea1cbf684f2b4fc2b5329610334d4762/users/e930e2400bac47c481332a72857166bb/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:44:23.422753 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd22b353-42b0-4a04-8755-b94889aff3e9 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:23.455376 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-26dcb109-3781-42b8-a2dc-aef6301e2b9f None None] MFA Rules not processed for user `cdfa309c37d943b992f6ca1c647557af`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:23.456945 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-26dcb109-3781-42b8-a2dc-aef6301e2b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:23.509006 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd22b353-42b0-4a04-8755-b94889aff3e9 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:23.515611 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a077956c-e3ed-43bb-b306-dd98b9be2845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:23.518143 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a077956c-e3ed-43bb-b306-dd98b9be2845 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:23.522191 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a077956c-e3ed-43bb-b306-dd98b9be2845 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:23.522553 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a077956c-e3ed-43bb-b306-dd98b9be2845 admin admin] PATH_INFO: `/v3/projects/ca38c1109acc49849daeb555b673e756/users/060d1fae9b0b488da1f7553cabd97d28/roles/3248f0268738489b883041851d681bb7` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:23.539423 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 641/1281] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:44:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:23.539630 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a077956c-e3ed-43bb-b306-dd98b9be2845 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca38c1109acc49849daeb555b673e756, user_id=060d1fae9b0b488da1f7553cabd97d28, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:23.540914 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dcb109-3781-42b8-a2dc-aef6301e2b9f None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:23.544501 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a077956c-e3ed-43bb-b306-dd98b9be2845 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:23.545056 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-26dcb109-3781-42b8-a2dc-aef6301e2b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:23.567616 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 641/1282] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:44:23 2022] PUT /identity/v3/projects/ca38c1109acc49849daeb555b673e756/users/060d1fae9b0b488da1f7553cabd97d28/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:44:23.574551 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a077956c-e3ed-43bb-b306-dd98b9be2845 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:23.636097 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:44:23.644703 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dcb109-3781-42b8-a2dc-aef6301e2b9f None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:23.667291 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a077956c-e3ed-43bb-b306-dd98b9be2845 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:23.672952 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc06c7c-38ec-47a2-9e27-a781386f5b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:23.673746 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc06c7c-38ec-47a2-9e27-a781386f5b03 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:23.674334 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc06c7c-38ec-47a2-9e27-a781386f5b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:23.674686 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc06c7c-38ec-47a2-9e27-a781386f5b03 admin admin] PATH_INFO: `/v3/projects/f67051bce2414ff791a2a1e1a638cadc/users/9cc9e2cf11804cc39a7de3c314e46679/roles/3248f0268738489b883041851d681bb7` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:23.692892 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc06c7c-38ec-47a2-9e27-a781386f5b03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f67051bce2414ff791a2a1e1a638cadc, user_id=9cc9e2cf11804cc39a7de3c314e46679, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:23.697361 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc06c7c-38ec-47a2-9e27-a781386f5b03 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:23.717768 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 642/1283] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:44:23 2022] PUT /identity/v3/projects/f67051bce2414ff791a2a1e1a638cadc/users/9cc9e2cf11804cc39a7de3c314e46679/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:44:23.718645 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc06c7c-38ec-47a2-9e27-a781386f5b03 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:23.733541 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18310eb-b8ae-4f2e-a182-61f85186d82f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:23.734439 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18310eb-b8ae-4f2e-a182-61f85186d82f service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:23.735236 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18310eb-b8ae-4f2e-a182-61f85186d82f service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:23.735521 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18310eb-b8ae-4f2e-a182-61f85186d82f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:23.760108 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18310eb-b8ae-4f2e-a182-61f85186d82f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:23.761653 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18310eb-b8ae-4f2e-a182-61f85186d82f service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:23.780521 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:44:23.781869 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 642/1284] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:44:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:44:23.783898 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e8f935-0746-4faf-8a00-ec58fdfe22c8 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:23.784340 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e8f935-0746-4faf-8a00-ec58fdfe22c8 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:23.784829 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e8f935-0746-4faf-8a00-ec58fdfe22c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:23.816920 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-34e8f935-0746-4faf-8a00-ec58fdfe22c8 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:23.818176 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-34e8f935-0746-4faf-8a00-ec58fdfe22c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:23.844788 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc06c7c-38ec-47a2-9e27-a781386f5b03 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:23.850221 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac376a78-1866-492c-85a2-103fc64ec30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:23.850921 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac376a78-1866-492c-85a2-103fc64ec30d admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:23.851254 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac376a78-1866-492c-85a2-103fc64ec30d admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:23.851586 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac376a78-1866-492c-85a2-103fc64ec30d admin admin] PATH_INFO: `/v3/projects/1df92a7d96684bed87c38a18e416c0e4/users/6cadc5fb6f834c04925b54e48296e668/roles/3248f0268738489b883041851d681bb7` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:23.872324 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac376a78-1866-492c-85a2-103fc64ec30d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1df92a7d96684bed87c38a18e416c0e4, user_id=6cadc5fb6f834c04925b54e48296e668, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:23.879836 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac376a78-1866-492c-85a2-103fc64ec30d admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:23.907919 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 643/1285] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:44:23 2022] PUT /identity/v3/projects/1df92a7d96684bed87c38a18e416c0e4/users/6cadc5fb6f834c04925b54e48296e668/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:44:23.910663 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab5db0c-e323-464a-9fac-ffc3c3f28119 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:23.912033 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab5db0c-e323-464a-9fac-ffc3c3f28119 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:23.913310 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab5db0c-e323-464a-9fac-ffc3c3f28119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:23.939121 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 643/1286] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:44:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:23.940849 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a7d83-64fa-45f5-845c-a9a8db761abf None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:23.943866 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a7d83-64fa-45f5-845c-a9a8db761abf None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:23.944166 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a7d83-64fa-45f5-845c-a9a8db761abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:23.964682 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-aab5db0c-e323-464a-9fac-ffc3c3f28119 None None] MFA Rules not processed for user `e930e2400bac47c481332a72857166bb`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:23.966336 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-aab5db0c-e323-464a-9fac-ffc3c3f28119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:23.987637 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-201a7d83-64fa-45f5-845c-a9a8db761abf None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:23.990003 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-201a7d83-64fa-45f5-845c-a9a8db761abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:24.111426 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 644/1287] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:44:23 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:24.112928 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8808a7-c3f8-40bf-b905-c2f1f4c7de54 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:24.113464 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8808a7-c3f8-40bf-b905-c2f1f4c7de54 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:24.113773 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8808a7-c3f8-40bf-b905-c2f1f4c7de54 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:24.115487 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 645/1288] 10.222.0.30 () {54 vars in 961 bytes} [Sat Oct 15 08:44:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:44:24.116536 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b239d81e-6749-40d6-b314-2a4a99ba4346 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:24.116817 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b239d81e-6749-40d6-b314-2a4a99ba4346 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:24.117092 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b239d81e-6749-40d6-b314-2a4a99ba4346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:24.133052 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 644/1289] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:44:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:24.134496 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c1bf80-c30a-4e79-bf67-36def318f244 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:24.134786 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c1bf80-c30a-4e79-bf67-36def318f244 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:24.136491 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c1bf80-c30a-4e79-bf67-36def318f244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:24.148428 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-b239d81e-6749-40d6-b314-2a4a99ba4346 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:24.150121 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-b239d81e-6749-40d6-b314-2a4a99ba4346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:24.169043 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 646/1290] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:44:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:24.170003 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b239d81e-6749-40d6-b314-2a4a99ba4346 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:24.183492 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-e6c1bf80-c30a-4e79-bf67-36def318f244 None None] MFA Rules not processed for user `6cadc5fb6f834c04925b54e48296e668`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:24.184848 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-e6c1bf80-c30a-4e79-bf67-36def318f244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:24.236841 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:44:24.263333 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b239d81e-6749-40d6-b314-2a4a99ba4346 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:24.270974 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d1838f-79c4-4665-b431-efad8d2437a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:24.275778 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d1838f-79c4-4665-b431-efad8d2437a4 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:24.275778 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d1838f-79c4-4665-b431-efad8d2437a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:24.275778 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d1838f-79c4-4665-b431-efad8d2437a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:24.370343 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 645/1291] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:44:24 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:24.370343 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c1bf80-c30a-4e79-bf67-36def318f244 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:24.371077 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d1838f-79c4-4665-b431-efad8d2437a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:24.371077 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d1838f-79c4-4665-b431-efad8d2437a4 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:24.371361 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c1bf80-c30a-4e79-bf67-36def318f244 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:24.371450 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 647/1292] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:44:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:44:24.371539 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88fcb5a-0e60-42d6-8415-fb7f91dad48d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:24.371629 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6052a6-000f-49bd-a5f5-aa2bc6cd6eb9 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:24.371629 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6052a6-000f-49bd-a5f5-aa2bc6cd6eb9 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:24.371787 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88fcb5a-0e60-42d6-8415-fb7f91dad48d admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:24.371899 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6052a6-000f-49bd-a5f5-aa2bc6cd6eb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:24.371982 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88fcb5a-0e60-42d6-8415-fb7f91dad48d admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:24.371982 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88fcb5a-0e60-42d6-8415-fb7f91dad48d admin admin] PATH_INFO: `/v3/projects` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:24.371982 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88fcb5a-0e60-42d6-8415-fb7f91dad48d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:24.371982 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88fcb5a-0e60-42d6-8415-fb7f91dad48d admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:24.406182 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 646/1293] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 08:44:24 2022] POST /identity/v3/projects => generated 403 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:44:24.407239 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88fcb5a-0e60-42d6-8415-fb7f91dad48d admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:24.433621 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-0e6052a6-000f-49bd-a5f5-aa2bc6cd6eb9 None None] MFA Rules not processed for user `af0a8af5cadc4cc1ab73e8eb2a20bb7e`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:24.435946 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-0e6052a6-000f-49bd-a5f5-aa2bc6cd6eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:24.478836 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:44:24.534879 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88fcb5a-0e60-42d6-8415-fb7f91dad48d admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:24.549900 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd23e51-45e9-4bbd-a87b-7197f7f62ff4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:24.550619 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd23e51-45e9-4bbd-a87b-7197f7f62ff4 service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:24.550997 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd23e51-45e9-4bbd-a87b-7197f7f62ff4 service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:24.551375 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd23e51-45e9-4bbd-a87b-7197f7f62ff4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:24.632623 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 648/1294] 10.222.0.30 () {58 vars in 1058 bytes} [Sat Oct 15 08:44:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:24.634392 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6052a6-000f-49bd-a5f5-aa2bc6cd6eb9 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:24.654163 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6052a6-000f-49bd-a5f5-aa2bc6cd6eb9 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:24.659053 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba86f99-81ad-4b73-8fd1-c5d924da0dd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:24.659716 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba86f99-81ad-4b73-8fd1-c5d924da0dd6 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:24.659999 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba86f99-81ad-4b73-8fd1-c5d924da0dd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:24.660270 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba86f99-81ad-4b73-8fd1-c5d924da0dd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:24.660391 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd23e51-45e9-4bbd-a87b-7197f7f62ff4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:24.662288 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd23e51-45e9-4bbd-a87b-7197f7f62ff4 service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:24.672004 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba86f99-81ad-4b73-8fd1-c5d924da0dd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:24.673950 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba86f99-81ad-4b73-8fd1-c5d924da0dd6 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:24.676458 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 647/1295] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:44:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:44:24.678099 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd23e51-45e9-4bbd-a87b-7197f7f62ff4 service neutron] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:24.686663 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 649/1296] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:44:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:44:24.687572 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba86f99-81ad-4b73-8fd1-c5d924da0dd6 service neutron] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:24.702314 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba86f99-81ad-4b73-8fd1-c5d924da0dd6 service neutron] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:24.709141 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5394129-e4f7-446e-9c35-bcb84ec4d5a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:24.709785 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5394129-e4f7-446e-9c35-bcb84ec4d5a9 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:24.710067 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5394129-e4f7-446e-9c35-bcb84ec4d5a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:24.710341 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5394129-e4f7-446e-9c35-bcb84ec4d5a9 admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:24.711158 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5394129-e4f7-446e-9c35-bcb84ec4d5a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:24.714078 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5394129-e4f7-446e-9c35-bcb84ec4d5a9 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:24.754998 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 650/1297] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:44:24 2022] POST /identity/v3/users => generated 308 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:44:24.756143 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5394129-e4f7-446e-9c35-bcb84ec4d5a9 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:24.758142 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd23e51-45e9-4bbd-a87b-7197f7f62ff4 service neutron] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:24.764236 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad8c523-f918-46ca-9814-74a6cd233a3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:24.764945 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad8c523-f918-46ca-9814-74a6cd233a3b service cinder] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:24.765232 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad8c523-f918-46ca-9814-74a6cd233a3b service cinder] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:24.765671 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad8c523-f918-46ca-9814-74a6cd233a3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:24.784611 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5394129-e4f7-446e-9c35-bcb84ec4d5a9 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:24.791847 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:44:24.792159 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2e90ae-c6f0-448f-ad42-01ac79ae5153 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0a8af5cadc4cc1ab73e8eb2a20bb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:24.797176 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e90ae-c6f0-448f-ad42-01ac79ae5153 service nova] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:24.797791 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e90ae-c6f0-448f-ad42-01ac79ae5153 service nova] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:24.798316 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e90ae-c6f0-448f-ad42-01ac79ae5153 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:24.845356 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad8c523-f918-46ca-9814-74a6cd233a3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:24.846874 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad8c523-f918-46ca-9814-74a6cd233a3b service cinder] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:24.857155 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 648/1298] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:44:24 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:44:24.857948 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad8c523-f918-46ca-9814-74a6cd233a3b service cinder] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:24.889127 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad8c523-f918-46ca-9814-74a6cd233a3b service cinder] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:24.897135 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c6174d-42a4-46c4-8699-8023866259ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:24.898181 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c6174d-42a4-46c4-8699-8023866259ef admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:24.899011 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c6174d-42a4-46c4-8699-8023866259ef admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:24.900070 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c6174d-42a4-46c4-8699-8023866259ef admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:24.901324 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c6174d-42a4-46c4-8699-8023866259ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:24.904447 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c6174d-42a4-46c4-8699-8023866259ef admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:24.927530 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 649/1299] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:44:24 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:44:24.927530 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c6174d-42a4-46c4-8699-8023866259ef admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:24.942957 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c6174d-42a4-46c4-8699-8023866259ef admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:24.973900 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2e90ae-c6f0-448f-ad42-01ac79ae5153 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:24.974469 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b2c01e-7cf5-4056-acea-1a8a0dad9264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:24.978240 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b2c01e-7cf5-4056-acea-1a8a0dad9264 admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:24.979366 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b2c01e-7cf5-4056-acea-1a8a0dad9264 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:24.980825 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b2c01e-7cf5-4056-acea-1a8a0dad9264 admin admin] PATH_INFO: `/v3/projects/2299065c2f7646a498190e537c036da0/users/badd528b366d4b61a1b12b7f780133bd/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:24.983399 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2e90ae-c6f0-448f-ad42-01ac79ae5153 service nova] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:25.023856 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 651/1300] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:44:24 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:44:25.035311 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b2c01e-7cf5-4056-acea-1a8a0dad9264 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2299065c2f7646a498190e537c036da0, user_id=badd528b366d4b61a1b12b7f780133bd, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:25.038843 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b2c01e-7cf5-4056-acea-1a8a0dad9264 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:25.064962 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 650/1301] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:44:24 2022] PUT /identity/v3/projects/2299065c2f7646a498190e537c036da0/users/badd528b366d4b61a1b12b7f780133bd/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:44:25.069248 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2e90ae-c6f0-448f-ad42-01ac79ae5153 service nova] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:25.083123 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2e90ae-c6f0-448f-ad42-01ac79ae5153 service nova] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:25.193881 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afcfd2b-936a-4eb4-b233-de03db5b4a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:25.194693 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afcfd2b-936a-4eb4-b233-de03db5b4a23 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:25.195244 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afcfd2b-936a-4eb4-b233-de03db5b4a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:25.195617 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afcfd2b-936a-4eb4-b233-de03db5b4a23 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:25.197224 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afcfd2b-936a-4eb4-b233-de03db5b4a23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:25.198972 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afcfd2b-936a-4eb4-b233-de03db5b4a23 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:25.216003 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 652/1302] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:44:25 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:44:25.220280 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b2c01e-7cf5-4056-acea-1a8a0dad9264 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:25.242749 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b2c01e-7cf5-4056-acea-1a8a0dad9264 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:25.266699 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9758b8-b8d1-4b6c-9cd8-4c7f15da1cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:25.269544 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9758b8-b8d1-4b6c-9cd8-4c7f15da1cdb admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:25.269830 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9758b8-b8d1-4b6c-9cd8-4c7f15da1cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:25.270162 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9758b8-b8d1-4b6c-9cd8-4c7f15da1cdb admin admin] PATH_INFO: `/v3/projects/2299065c2f7646a498190e537c036da0/users/badd528b366d4b61a1b12b7f780133bd/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:25.273344 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9758b8-b8d1-4b6c-9cd8-4c7f15da1cdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2299065c2f7646a498190e537c036da0, user_id=badd528b366d4b61a1b12b7f780133bd, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:25.280859 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9758b8-b8d1-4b6c-9cd8-4c7f15da1cdb admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:25.318879 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 651/1303] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:44:25 2022] PUT /identity/v3/projects/2299065c2f7646a498190e537c036da0/users/badd528b366d4b61a1b12b7f780133bd/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:44:25.322066 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e457751-582d-49d9-bf96-a230611f73a5 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:25.322466 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e457751-582d-49d9-bf96-a230611f73a5 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:25.322928 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e457751-582d-49d9-bf96-a230611f73a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:25.391842 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:44:25.402742 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-2e457751-582d-49d9-bf96-a230611f73a5 None None] MFA Rules not processed for user `badd528b366d4b61a1b12b7f780133bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:25.404927 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-2e457751-582d-49d9-bf96-a230611f73a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:25.510878 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6380a7-79e9-49a8-bc7c-686a0b6b0600 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:25.511135 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6380a7-79e9-49a8-bc7c-686a0b6b0600 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:25.511418 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6380a7-79e9-49a8-bc7c-686a0b6b0600 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:25.513016 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 652/1304] 10.222.0.30 () {54 vars in 912 bytes} [Sat Oct 15 08:44:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:44:25.519558 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ef60c-1d77-4102-9ecc-948c6a1b15fd None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:25.519841 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ef60c-1d77-4102-9ecc-948c6a1b15fd None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:25.520123 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ef60c-1d77-4102-9ecc-948c6a1b15fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:25.553161 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 653/1305] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:44:25 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:25.554784 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-0a1ef60c-1d77-4102-9ecc-948c6a1b15fd None None] MFA Rules not processed for user `af0a8af5cadc4cc1ab73e8eb2a20bb7e`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:25.556345 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-0a1ef60c-1d77-4102-9ecc-948c6a1b15fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:25.566150 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e457751-582d-49d9-bf96-a230611f73a5 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:25.705114 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 653/1306] 10.222.0.30 () {58 vars in 1009 bytes} [Sat Oct 15 08:44:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:25.746620 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e457751-582d-49d9-bf96-a230611f73a5 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:25.777699 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1ef60c-1d77-4102-9ecc-948c6a1b15fd None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:25.789226 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082778db-fca6-4c27-b4a8-16af39effea8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:25.790053 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082778db-fca6-4c27-b4a8-16af39effea8 service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:25.790436 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082778db-fca6-4c27-b4a8-16af39effea8 service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:25.790811 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082778db-fca6-4c27-b4a8-16af39effea8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:25.805305 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082778db-fca6-4c27-b4a8-16af39effea8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:25.808694 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082778db-fca6-4c27-b4a8-16af39effea8 service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:25.810277 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1ef60c-1d77-4102-9ecc-948c6a1b15fd None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:25.826311 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1438bdfe-dab4-4d7c-9ce4-032ebd653497 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:25.826548 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 654/1307] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:44:25 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:44:25.827420 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1438bdfe-dab4-4d7c-9ce4-032ebd653497 service cinder] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:25.829799 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1438bdfe-dab4-4d7c-9ce4-032ebd653497 service cinder] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:25.831927 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1438bdfe-dab4-4d7c-9ce4-032ebd653497 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:25.861876 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1438bdfe-dab4-4d7c-9ce4-032ebd653497 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:25.867597 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1438bdfe-dab4-4d7c-9ce4-032ebd653497 service cinder] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:25.907955 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 654/1308] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:44:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 08:44:26.504237 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082778db-fca6-4c27-b4a8-16af39effea8 service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:26.770419 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082778db-fca6-4c27-b4a8-16af39effea8 service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:26.790315 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fb3169-b24a-4862-b7d4-390120a2fa6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:26.791143 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb3169-b24a-4862-b7d4-390120a2fa6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:26.792077 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb3169-b24a-4862-b7d4-390120a2fa6d admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:26.792077 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb3169-b24a-4862-b7d4-390120a2fa6d admin admin] PATH_INFO: `/v3/users/e930e2400bac47c481332a72857166bb` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:26.795547 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fb3169-b24a-4862-b7d4-390120a2fa6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e930e2400bac47c481332a72857166bb)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:26.800015 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fb3169-b24a-4862-b7d4-390120a2fa6d admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:26.898377 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:44:26.904970 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-37fb3169-b24a-4862-b7d4-390120a2fa6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e930e2400bac47c481332a72857166bb'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:26.907717 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-37fb3169-b24a-4862-b7d4-390120a2fa6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e930e2400bac47c481332a72857166bb'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:26.927818 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-37fb3169-b24a-4862-b7d4-390120a2fa6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e930e2400bac47c481332a72857166bb'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:26.958021 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-37fb3169-b24a-4862-b7d4-390120a2fa6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e930e2400bac47c481332a72857166bb'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:26.975808 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-37fb3169-b24a-4862-b7d4-390120a2fa6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e930e2400bac47c481332a72857166bb'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:26.989356 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ce639-8397-4ec2-98dc-3159cb9d7184 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:26.989986 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ce639-8397-4ec2-98dc-3159cb9d7184 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:26.990354 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ce639-8397-4ec2-98dc-3159cb9d7184 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:26.992716 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 655/1309] 10.222.0.30 () {54 vars in 961 bytes} [Sat Oct 15 08:44:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:44:26.994783 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 655/1310] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 08:44:26 2022] DELETE /identity/v3/users/e930e2400bac47c481332a72857166bb => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:44:26.999094 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958d0af4-76c9-4c01-ae53-b6a3533bca02 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:26.999610 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958d0af4-76c9-4c01-ae53-b6a3533bca02 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:27.000014 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958d0af4-76c9-4c01-ae53-b6a3533bca02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:27.008821 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a251b0-c718-495c-86c8-0836d2e15661 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:27.009198 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a251b0-c718-495c-86c8-0836d2e15661 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:27.009315 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a251b0-c718-495c-86c8-0836d2e15661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:27.033199 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-958d0af4-76c9-4c01-ae53-b6a3533bca02 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:27.035491 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-958d0af4-76c9-4c01-ae53-b6a3533bca02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:27.043256 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-03a251b0-c718-495c-86c8-0836d2e15661 None None] MFA Rules not processed for user `af0a8af5cadc4cc1ab73e8eb2a20bb7e`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:27.045537 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-03a251b0-c718-495c-86c8-0836d2e15661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:27.124447 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:44:27.151765 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 656/1311] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:44:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:27.190049 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958d0af4-76c9-4c01-ae53-b6a3533bca02 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:27.191447 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-958d0af4-76c9-4c01-ae53-b6a3533bca02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:27.194897 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 656/1312] 10.222.0.30 () {58 vars in 1058 bytes} [Sat Oct 15 08:44:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 08:44:27.200921 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169254c2-b7da-4787-aeaf-fe51ea0f08e3 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:27.202318 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169254c2-b7da-4787-aeaf-fe51ea0f08e3 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:27.202822 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169254c2-b7da-4787-aeaf-fe51ea0f08e3 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:27.204739 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 657/1313] 10.222.0.30 () {54 vars in 912 bytes} [Sat Oct 15 08:44:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:44:27.211702 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165089cf-2ee8-4397-94d7-c93e45cb6033 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:27.213886 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165089cf-2ee8-4397-94d7-c93e45cb6033 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:27.214363 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165089cf-2ee8-4397-94d7-c93e45cb6033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:27.252847 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-165089cf-2ee8-4397-94d7-c93e45cb6033 None None] MFA Rules not processed for user `af0a8af5cadc4cc1ab73e8eb2a20bb7e`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:27.254413 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-165089cf-2ee8-4397-94d7-c93e45cb6033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:27.280339 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 658/1314] 10.222.0.30 () {58 vars in 1009 bytes} [Sat Oct 15 08:44:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:27.290572 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165089cf-2ee8-4397-94d7-c93e45cb6033 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:27.293185 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-165089cf-2ee8-4397-94d7-c93e45cb6033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:27.314392 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958d0af4-76c9-4c01-ae53-b6a3533bca02 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:27.325831 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a908b29-ed57-4c79-b32b-108cb17401aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:27.329703 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a908b29-ed57-4c79-b32b-108cb17401aa service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:27.330024 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a908b29-ed57-4c79-b32b-108cb17401aa service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:27.330388 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a908b29-ed57-4c79-b32b-108cb17401aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:27.357965 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a908b29-ed57-4c79-b32b-108cb17401aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:27.360294 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a908b29-ed57-4c79-b32b-108cb17401aa service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:27.420573 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 657/1315] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:44:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:44:27.459854 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165089cf-2ee8-4397-94d7-c93e45cb6033 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:27.469630 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6c6447-16fb-40a1-b501-7fe07d62f81a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:27.470297 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6c6447-16fb-40a1-b501-7fe07d62f81a service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:27.470586 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6c6447-16fb-40a1-b501-7fe07d62f81a service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:27.473429 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6c6447-16fb-40a1-b501-7fe07d62f81a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:27.488735 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6c6447-16fb-40a1-b501-7fe07d62f81a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:27.490526 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6c6447-16fb-40a1-b501-7fe07d62f81a service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:27.510148 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 659/1316] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:44:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:44:29.171426 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a908b29-ed57-4c79-b32b-108cb17401aa service neutron] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:29.174299 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-9a908b29-ed57-4c79-b32b-108cb17401aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:29.200001 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a908b29-ed57-4c79-b32b-108cb17401aa service neutron] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:29.215278 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501de48c-576c-4297-ae37-2c8e28f3ef96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:29.216427 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501de48c-576c-4297-ae37-2c8e28f3ef96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:29.216860 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501de48c-576c-4297-ae37-2c8e28f3ef96 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:29.217292 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501de48c-576c-4297-ae37-2c8e28f3ef96 admin admin] PATH_INFO: `/v3/projects/ea1cbf684f2b4fc2b5329610334d4762` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:29.220434 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501de48c-576c-4297-ae37-2c8e28f3ef96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea1cbf684f2b4fc2b5329610334d4762)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:29.223161 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501de48c-576c-4297-ae37-2c8e28f3ef96 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:29.361365 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-501de48c-576c-4297-ae37-2c8e28f3ef96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea1cbf684f2b4fc2b5329610334d4762', 'request_id': 'req-501de48c-576c-4297-ae37-2c8e28f3ef96'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:29.374795 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-501de48c-576c-4297-ae37-2c8e28f3ef96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea1cbf684f2b4fc2b5329610334d4762', 'request_id': 'req-501de48c-576c-4297-ae37-2c8e28f3ef96'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:29.386095 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-501de48c-576c-4297-ae37-2c8e28f3ef96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea1cbf684f2b4fc2b5329610334d4762', 'request_id': 'req-501de48c-576c-4297-ae37-2c8e28f3ef96'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:29.411326 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-501de48c-576c-4297-ae37-2c8e28f3ef96 admin admin] The token cache is being invalidate because project ea1cbf684f2b4fc2b5329610334d4762 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 08:44:29.412468 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-501de48c-576c-4297-ae37-2c8e28f3ef96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:29.413533 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-501de48c-576c-4297-ae37-2c8e28f3ef96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:29.415781 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 658/1317] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 08:44:29 2022] DELETE /identity/v3/projects/ea1cbf684f2b4fc2b5329610334d4762 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:44:29.419599 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6c6447-16fb-40a1-b501-7fe07d62f81a service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:29.450525 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6c6447-16fb-40a1-b501-7fe07d62f81a service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:29.551130 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:44:29.591695 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd909054-377b-4672-b87d-3f3c3121fdb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:29.592220 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd909054-377b-4672-b87d-3f3c3121fdb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:29.592507 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd909054-377b-4672-b87d-3f3c3121fdb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:29.592783 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd909054-377b-4672-b87d-3f3c3121fdb5 admin admin] PATH_INFO: `/v3/users/badd528b366d4b61a1b12b7f780133bd` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:29.595312 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd909054-377b-4672-b87d-3f3c3121fdb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=badd528b366d4b61a1b12b7f780133bd)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:29.602126 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd909054-377b-4672-b87d-3f3c3121fdb5 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:29.722586 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-bd909054-377b-4672-b87d-3f3c3121fdb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'badd528b366d4b61a1b12b7f780133bd'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:29.730327 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-bd909054-377b-4672-b87d-3f3c3121fdb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'badd528b366d4b61a1b12b7f780133bd'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:29.750835 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-bd909054-377b-4672-b87d-3f3c3121fdb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'badd528b366d4b61a1b12b7f780133bd'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:29.801048 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-bd909054-377b-4672-b87d-3f3c3121fdb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'badd528b366d4b61a1b12b7f780133bd'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:29.823674 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-bd909054-377b-4672-b87d-3f3c3121fdb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'badd528b366d4b61a1b12b7f780133bd'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:29.835289 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 660/1318] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 08:44:29 2022] DELETE /identity/v3/users/badd528b366d4b61a1b12b7f780133bd => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:44:31.811704 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501de48c-576c-4297-ae37-2c8e28f3ef96 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:31.813375 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-501de48c-576c-4297-ae37-2c8e28f3ef96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:31.841768 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501de48c-576c-4297-ae37-2c8e28f3ef96 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:31.986136 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a089d778-c4bc-4370-91fd-124a3c4519c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:31.989432 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a089d778-c4bc-4370-91fd-124a3c4519c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:31.989978 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a089d778-c4bc-4370-91fd-124a3c4519c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:31.990530 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a089d778-c4bc-4370-91fd-124a3c4519c1 admin admin] PATH_INFO: `/v3/projects/2299065c2f7646a498190e537c036da0` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:31.992560 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a089d778-c4bc-4370-91fd-124a3c4519c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2299065c2f7646a498190e537c036da0)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:32.002490 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a089d778-c4bc-4370-91fd-124a3c4519c1 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:32.049343 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:44:32.168212 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-a089d778-c4bc-4370-91fd-124a3c4519c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2299065c2f7646a498190e537c036da0', 'request_id': 'req-a089d778-c4bc-4370-91fd-124a3c4519c1'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:32.214132 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-a089d778-c4bc-4370-91fd-124a3c4519c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2299065c2f7646a498190e537c036da0', 'request_id': 'req-a089d778-c4bc-4370-91fd-124a3c4519c1'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:32.229368 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-a089d778-c4bc-4370-91fd-124a3c4519c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2299065c2f7646a498190e537c036da0', 'request_id': 'req-a089d778-c4bc-4370-91fd-124a3c4519c1'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:32.276097 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-a089d778-c4bc-4370-91fd-124a3c4519c1 admin admin] The token cache is being invalidate because project 2299065c2f7646a498190e537c036da0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 08:44:32.276603 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-a089d778-c4bc-4370-91fd-124a3c4519c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:32.278045 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-a089d778-c4bc-4370-91fd-124a3c4519c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:32.282902 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 659/1319] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 08:44:31 2022] DELETE /identity/v3/projects/2299065c2f7646a498190e537c036da0 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:44:32.288489 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70665db-64a8-4829-8d3c-2efd4c995f13 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:32.289050 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70665db-64a8-4829-8d3c-2efd4c995f13 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:32.289050 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70665db-64a8-4829-8d3c-2efd4c995f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:32.351094 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-a70665db-64a8-4829-8d3c-2efd4c995f13 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:32.364056 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-a70665db-64a8-4829-8d3c-2efd4c995f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:32.593651 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab13ed7-25ac-4593-a1aa-dba0e2acc38b None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:32.594031 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab13ed7-25ac-4593-a1aa-dba0e2acc38b None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:32.594244 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab13ed7-25ac-4593-a1aa-dba0e2acc38b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:32.700107 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 661/1320] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:44:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:32.702166 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-fab13ed7-25ac-4593-a1aa-dba0e2acc38b None None] MFA Rules not processed for user `a5d85f7efca040cc81253dfbf0de7c42`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:32.709235 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cdbcf9-9d4b-4408-b785-3fcac385c8d2 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:32.710851 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-fab13ed7-25ac-4593-a1aa-dba0e2acc38b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:32.711208 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cdbcf9-9d4b-4408-b785-3fcac385c8d2 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:32.712225 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cdbcf9-9d4b-4408-b785-3fcac385c8d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:32.765274 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-45cdbcf9-9d4b-4408-b785-3fcac385c8d2 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:32.768108 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-45cdbcf9-9d4b-4408-b785-3fcac385c8d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:32.803144 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 662/1321] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:44:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:32.807954 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cdbcf9-9d4b-4408-b785-3fcac385c8d2 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:32.827660 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cdbcf9-9d4b-4408-b785-3fcac385c8d2 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:32.843902 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef4a4d9-fcc9-4c1f-83a3-d230f850b24b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:32.845110 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef4a4d9-fcc9-4c1f-83a3-d230f850b24b admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:32.845602 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef4a4d9-fcc9-4c1f-83a3-d230f850b24b admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:32.846173 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef4a4d9-fcc9-4c1f-83a3-d230f850b24b admin admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:32.847052 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef4a4d9-fcc9-4c1f-83a3-d230f850b24b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:44:32.849345 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef4a4d9-fcc9-4c1f-83a3-d230f850b24b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:32.854018 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef4a4d9-fcc9-4c1f-83a3-d230f850b24b admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:32.887783 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 663/1322] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 08:44:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 08:44:32.898849 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef4a4d9-fcc9-4c1f-83a3-d230f850b24b admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:32.915784 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef4a4d9-fcc9-4c1f-83a3-d230f850b24b admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:32.933080 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f4584e-f2de-462d-90f4-03785d2f0bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:32.933914 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f4584e-f2de-462d-90f4-03785d2f0bbd admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:32.934300 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f4584e-f2de-462d-90f4-03785d2f0bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:32.934693 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f4584e-f2de-462d-90f4-03785d2f0bbd admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:32.935522 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f4584e-f2de-462d-90f4-03785d2f0bbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:32.937545 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f4584e-f2de-462d-90f4-03785d2f0bbd admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:32.942569 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 660/1323] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:44:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:32.956893 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab13ed7-25ac-4593-a1aa-dba0e2acc38b None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:32.959591 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-fab13ed7-25ac-4593-a1aa-dba0e2acc38b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:33.014366 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 664/1324] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 08:44:32 2022] POST /identity/v3/projects => generated 363 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:44:33.022707 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f4584e-f2de-462d-90f4-03785d2f0bbd admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:33.090802 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:44:33.150174 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f4584e-f2de-462d-90f4-03785d2f0bbd admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:33.157952 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2ebd4a-7476-4658-a580-1283dbc2ac9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:33.158773 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ebd4a-7476-4658-a580-1283dbc2ac9a admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:33.159143 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ebd4a-7476-4658-a580-1283dbc2ac9a admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:33.159519 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ebd4a-7476-4658-a580-1283dbc2ac9a admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:33.160263 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2ebd4a-7476-4658-a580-1283dbc2ac9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:33.162889 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2ebd4a-7476-4658-a580-1283dbc2ac9a admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:33.209666 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 665/1325] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:44:33 2022] POST /identity/v3/users => generated 282 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:44:33.211627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2ebd4a-7476-4658-a580-1283dbc2ac9a admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:33.236102 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2ebd4a-7476-4658-a580-1283dbc2ac9a admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:33.251424 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab13ed7-25ac-4593-a1aa-dba0e2acc38b None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:33.253740 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb017ee-f3a4-4943-81e1-e079604dd74f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:33.254998 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb017ee-f3a4-4943-81e1-e079604dd74f admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:33.255483 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb017ee-f3a4-4943-81e1-e079604dd74f admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:33.255865 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb017ee-f3a4-4943-81e1-e079604dd74f admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:33.256723 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb017ee-f3a4-4943-81e1-e079604dd74f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:33.259289 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb017ee-f3a4-4943-81e1-e079604dd74f admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:33.283615 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 666/1326] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:44:33 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:44:33.291452 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb017ee-f3a4-4943-81e1-e079604dd74f admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:33.314287 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb017ee-f3a4-4943-81e1-e079604dd74f admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:33.327150 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:44:33.337251 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3057cc-9294-456b-86ff-3a58f6be4c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:33.339969 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3057cc-9294-456b-86ff-3a58f6be4c6e admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:33.340488 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3057cc-9294-456b-86ff-3a58f6be4c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:33.341000 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3057cc-9294-456b-86ff-3a58f6be4c6e admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:33.341942 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3057cc-9294-456b-86ff-3a58f6be4c6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:33.344222 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3057cc-9294-456b-86ff-3a58f6be4c6e admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:33.384615 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 667/1327] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:44:33 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:44:33.391783 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3057cc-9294-456b-86ff-3a58f6be4c6e admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:33.413568 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3057cc-9294-456b-86ff-3a58f6be4c6e admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:33.424545 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d340e6e-9bc7-4943-8208-6f36f21af89a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:33.426036 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d340e6e-9bc7-4943-8208-6f36f21af89a admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:33.427120 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d340e6e-9bc7-4943-8208-6f36f21af89a admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:33.427640 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d340e6e-9bc7-4943-8208-6f36f21af89a admin admin] PATH_INFO: `/v3/projects/4f5a931619924d06985b4538a06ff9f9/users/caa93bd1fe3e4e6fb1fb6231d77b28ad/roles/3248f0268738489b883041851d681bb7` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:33.449908 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5485233c-0374-41f1-ba75-2aa67e65bc4b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0a8af5cadc4cc1ab73e8eb2a20bb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:33.450563 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5485233c-0374-41f1-ba75-2aa67e65bc4b service nova] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:33.455015 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5485233c-0374-41f1-ba75-2aa67e65bc4b service nova] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:33.455461 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5485233c-0374-41f1-ba75-2aa67e65bc4b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:33.472646 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d340e6e-9bc7-4943-8208-6f36f21af89a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f5a931619924d06985b4538a06ff9f9, user_id=caa93bd1fe3e4e6fb1fb6231d77b28ad, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:33.478430 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d340e6e-9bc7-4943-8208-6f36f21af89a admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:33.527616 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 668/1328] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:44:33 2022] PUT /identity/v3/projects/4f5a931619924d06985b4538a06ff9f9/users/caa93bd1fe3e4e6fb1fb6231d77b28ad/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:44:33.529539 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46fe507-8300-4dbb-92fa-6759876a395c None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:33.530056 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46fe507-8300-4dbb-92fa-6759876a395c None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:33.530546 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46fe507-8300-4dbb-92fa-6759876a395c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:33.590519 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-f46fe507-8300-4dbb-92fa-6759876a395c None None] MFA Rules not processed for user `caa93bd1fe3e4e6fb1fb6231d77b28ad`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:33.596684 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-f46fe507-8300-4dbb-92fa-6759876a395c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:33.650965 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5485233c-0374-41f1-ba75-2aa67e65bc4b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:33.652844 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5485233c-0374-41f1-ba75-2aa67e65bc4b service nova] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:33.672055 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 661/1329] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:44:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 715 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 08:44:33.760276 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 669/1330] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:44:33 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:33.765079 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5485233c-0374-41f1-ba75-2aa67e65bc4b service nova] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:33.781822 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5485233c-0374-41f1-ba75-2aa67e65bc4b service nova] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:33.964910 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b171c846-5f80-44a6-a8fb-454806d1fdad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:33.972168 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171c846-5f80-44a6-a8fb-454806d1fdad admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:33.973194 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171c846-5f80-44a6-a8fb-454806d1fdad admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:33.973602 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171c846-5f80-44a6-a8fb-454806d1fdad admin admin] PATH_INFO: `/v3/projects` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:33.974447 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b171c846-5f80-44a6-a8fb-454806d1fdad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:33.978482 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b171c846-5f80-44a6-a8fb-454806d1fdad admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:34.089075 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46fe507-8300-4dbb-92fa-6759876a395c None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:34.090213 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 662/1331] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 08:44:33 2022] POST /identity/v3/projects => generated 365 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:44:34.115062 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46fe507-8300-4dbb-92fa-6759876a395c None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:34.369643 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd64942-1401-452f-9d44-e37c4d78c004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:34.375667 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd64942-1401-452f-9d44-e37c4d78c004 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:34.376149 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd64942-1401-452f-9d44-e37c4d78c004 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:34.376536 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd64942-1401-452f-9d44-e37c4d78c004 admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:34.377327 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd64942-1401-452f-9d44-e37c4d78c004 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:34.379704 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd64942-1401-452f-9d44-e37c4d78c004 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:34.401160 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:44:34.439982 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b171c846-5f80-44a6-a8fb-454806d1fdad admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:34.449897 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 670/1332] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:44:34 2022] POST /identity/v3/users => generated 289 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:44:34.458306 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b171c846-5f80-44a6-a8fb-454806d1fdad admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:34.458550 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd64942-1401-452f-9d44-e37c4d78c004 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:34.468670 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-bdd64942-1401-452f-9d44-e37c4d78c004 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:34.478287 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859319cc-4bf7-41d0-a4cc-cb6e3f1b8de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:34.479117 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859319cc-4bf7-41d0-a4cc-cb6e3f1b8de5 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:34.482728 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859319cc-4bf7-41d0-a4cc-cb6e3f1b8de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:34.483276 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859319cc-4bf7-41d0-a4cc-cb6e3f1b8de5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:34.484202 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859319cc-4bf7-41d0-a4cc-cb6e3f1b8de5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:34.488715 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859319cc-4bf7-41d0-a4cc-cb6e3f1b8de5 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:34.512067 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd64942-1401-452f-9d44-e37c4d78c004 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:34.529473 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 663/1333] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:44:34 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:44:34.530815 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859319cc-4bf7-41d0-a4cc-cb6e3f1b8de5 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:34.539001 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d20d97c-4154-4b39-b55d-f993151232f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:34.550284 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859319cc-4bf7-41d0-a4cc-cb6e3f1b8de5 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:34.553829 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d20d97c-4154-4b39-b55d-f993151232f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:34.554315 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d20d97c-4154-4b39-b55d-f993151232f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:34.554613 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d20d97c-4154-4b39-b55d-f993151232f2 admin admin] PATH_INFO: `/v3/users/cdfa309c37d943b992f6ca1c647557af` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:34.557193 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d20d97c-4154-4b39-b55d-f993151232f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdfa309c37d943b992f6ca1c647557af)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:34.566901 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d20d97c-4154-4b39-b55d-f993151232f2 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:34.568175 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bdc1f7-3bfd-48d2-993a-a9a67608e586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:34.572120 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bdc1f7-3bfd-48d2-993a-a9a67608e586 admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:34.573670 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bdc1f7-3bfd-48d2-993a-a9a67608e586 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:34.574203 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bdc1f7-3bfd-48d2-993a-a9a67608e586 admin admin] PATH_INFO: `/v3/projects/a1a3585c814a4050996be4a854796598/users/ba3556a1334f46369dbaac93f52a8ff4/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:34.632941 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bdc1f7-3bfd-48d2-993a-a9a67608e586 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1a3585c814a4050996be4a854796598, user_id=ba3556a1334f46369dbaac93f52a8ff4, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:34.638343 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bdc1f7-3bfd-48d2-993a-a9a67608e586 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:34.675705 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 664/1334] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:44:34 2022] PUT /identity/v3/projects/a1a3585c814a4050996be4a854796598/users/ba3556a1334f46369dbaac93f52a8ff4/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:44:34.677863 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bdc1f7-3bfd-48d2-993a-a9a67608e586 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:34.747380 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-5d20d97c-4154-4b39-b55d-f993151232f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdfa309c37d943b992f6ca1c647557af'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:34.754107 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-5d20d97c-4154-4b39-b55d-f993151232f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdfa309c37d943b992f6ca1c647557af'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:34.765295 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:44:34.779726 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-5d20d97c-4154-4b39-b55d-f993151232f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdfa309c37d943b992f6ca1c647557af'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:34.807044 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-5d20d97c-4154-4b39-b55d-f993151232f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdfa309c37d943b992f6ca1c647557af'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:34.825584 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-5d20d97c-4154-4b39-b55d-f993151232f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdfa309c37d943b992f6ca1c647557af'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:34.836399 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 671/1335] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 08:44:34 2022] DELETE /identity/v3/users/cdfa309c37d943b992f6ca1c647557af => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:44:34.841586 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1e5cc9-4b9b-4e9b-b033-401b1e71c99f None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:34.841760 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1e5cc9-4b9b-4e9b-b033-401b1e71c99f None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:34.842049 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1e5cc9-4b9b-4e9b-b033-401b1e71c99f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:34.900116 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bdc1f7-3bfd-48d2-993a-a9a67608e586 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:34.914291 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-0c1e5cc9-4b9b-4e9b-b033-401b1e71c99f None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:34.921720 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-0c1e5cc9-4b9b-4e9b-b033-401b1e71c99f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:35.072408 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17d9442-8c59-4a9e-9fac-5eb33cb830f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:35.073616 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d9442-8c59-4a9e-9fac-5eb33cb830f3 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:35.074224 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d9442-8c59-4a9e-9fac-5eb33cb830f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:35.074679 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d9442-8c59-4a9e-9fac-5eb33cb830f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:35.075505 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17d9442-8c59-4a9e-9fac-5eb33cb830f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:35.078092 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17d9442-8c59-4a9e-9fac-5eb33cb830f3 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:35.114238 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 665/1336] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:44:34 2022] GET /identity/v3/roles => generated 1420 bytes in 438 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:44:35.119680 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17d9442-8c59-4a9e-9fac-5eb33cb830f3 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:35.123350 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 672/1337] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:44:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:35.125585 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-d17d9442-8c59-4a9e-9fac-5eb33cb830f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:35.150423 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17d9442-8c59-4a9e-9fac-5eb33cb830f3 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:35.163743 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab74bcb-72f3-46b8-a67c-500e552017d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:35.164978 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab74bcb-72f3-46b8-a67c-500e552017d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:35.165586 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab74bcb-72f3-46b8-a67c-500e552017d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:35.166038 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab74bcb-72f3-46b8-a67c-500e552017d8 admin admin] PATH_INFO: `/v3/projects/a1a3585c814a4050996be4a854796598/users/ba3556a1334f46369dbaac93f52a8ff4/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:35.172083 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab74bcb-72f3-46b8-a67c-500e552017d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1a3585c814a4050996be4a854796598, user_id=ba3556a1334f46369dbaac93f52a8ff4, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:35.175663 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab74bcb-72f3-46b8-a67c-500e552017d8 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:35.201303 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 666/1338] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:44:35 2022] PUT /identity/v3/projects/a1a3585c814a4050996be4a854796598/users/ba3556a1334f46369dbaac93f52a8ff4/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 08:44:35.213183 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7a564a-1302-411a-a5cb-14645525456a None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:35.213479 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7a564a-1302-411a-a5cb-14645525456a None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:35.213810 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7a564a-1302-411a-a5cb-14645525456a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:35.288034 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-3e7a564a-1302-411a-a5cb-14645525456a None None] MFA Rules not processed for user `ba3556a1334f46369dbaac93f52a8ff4`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:35.289308 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-3e7a564a-1302-411a-a5cb-14645525456a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:35.343650 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab74bcb-72f3-46b8-a67c-500e552017d8 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:35.346307 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-6ab74bcb-72f3-46b8-a67c-500e552017d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:35.533216 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab74bcb-72f3-46b8-a67c-500e552017d8 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:35.547499 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1684fc2-c7cf-4da4-986d-08ece006f9a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:35.548536 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1684fc2-c7cf-4da4-986d-08ece006f9a1 service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:35.549001 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1684fc2-c7cf-4da4-986d-08ece006f9a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:35.551425 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1684fc2-c7cf-4da4-986d-08ece006f9a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:35.714778 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1684fc2-c7cf-4da4-986d-08ece006f9a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:35.717716 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1684fc2-c7cf-4da4-986d-08ece006f9a1 service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:35.730520 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:44:35.744174 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 667/1339] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:44:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 08:44:35.806004 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:INSERT=4,keystone:SELECT=86,keystone:DELETE=10 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:44:35.860542 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 673/1340] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:44:35 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 649 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:35.873289 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1684fc2-c7cf-4da4-986d-08ece006f9a1 service neutron] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:35.874844 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-d1684fc2-c7cf-4da4-986d-08ece006f9a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:36.010130 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1684fc2-c7cf-4da4-986d-08ece006f9a1 service neutron] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:36.023102 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1abc20-88b2-4df3-a705-0e9e6a6ee4bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:36.024547 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1abc20-88b2-4df3-a705-0e9e6a6ee4bd service cinder] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:36.024857 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1abc20-88b2-4df3-a705-0e9e6a6ee4bd service cinder] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:36.025894 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1abc20-88b2-4df3-a705-0e9e6a6ee4bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:36.049734 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1abc20-88b2-4df3-a705-0e9e6a6ee4bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:36.051199 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1abc20-88b2-4df3-a705-0e9e6a6ee4bd service cinder] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:36.091618 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 668/1341] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:44:35 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:44:37.061669 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7a564a-1302-411a-a5cb-14645525456a None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:37.079884 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7a564a-1302-411a-a5cb-14645525456a None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:37.093851 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c1b1fb-978d-49ed-9774-2b7010ca8d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:37.094796 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c1b1fb-978d-49ed-9774-2b7010ca8d63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:37.095242 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c1b1fb-978d-49ed-9774-2b7010ca8d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:37.095668 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c1b1fb-978d-49ed-9774-2b7010ca8d63 admin admin] PATH_INFO: `/v3/users/caa93bd1fe3e4e6fb1fb6231d77b28ad` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:37.097747 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c1b1fb-978d-49ed-9774-2b7010ca8d63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=caa93bd1fe3e4e6fb1fb6231d77b28ad)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:37.104224 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c1b1fb-978d-49ed-9774-2b7010ca8d63 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:37.205881 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-71c1b1fb-978d-49ed-9774-2b7010ca8d63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'caa93bd1fe3e4e6fb1fb6231d77b28ad'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:37.206877 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-71c1b1fb-978d-49ed-9774-2b7010ca8d63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'caa93bd1fe3e4e6fb1fb6231d77b28ad'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:37.225241 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-71c1b1fb-978d-49ed-9774-2b7010ca8d63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'caa93bd1fe3e4e6fb1fb6231d77b28ad'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:37.249081 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-71c1b1fb-978d-49ed-9774-2b7010ca8d63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'caa93bd1fe3e4e6fb1fb6231d77b28ad'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:37.261553 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-71c1b1fb-978d-49ed-9774-2b7010ca8d63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'caa93bd1fe3e4e6fb1fb6231d77b28ad'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:37.270933 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 674/1342] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 08:44:37 2022] DELETE /identity/v3/users/caa93bd1fe3e4e6fb1fb6231d77b28ad => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:44:37.276244 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161618b5-cde5-4ec7-a1a0-a0d0d2ce5e79 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:37.277274 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161618b5-cde5-4ec7-a1a0-a0d0d2ce5e79 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:37.277797 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161618b5-cde5-4ec7-a1a0-a0d0d2ce5e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:37.296225 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c1b1fb-978d-49ed-9774-2b7010ca8d63 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:37.300210 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-71c1b1fb-978d-49ed-9774-2b7010ca8d63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:37.333701 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-161618b5-cde5-4ec7-a1a0-a0d0d2ce5e79 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:37.335467 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-161618b5-cde5-4ec7-a1a0-a0d0d2ce5e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:37.429150 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c1b1fb-978d-49ed-9774-2b7010ca8d63 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:37.442679 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740b9c86-71f5-4fd9-ac9f-9efcaa2edd73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:37.443509 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b9c86-71f5-4fd9-ac9f-9efcaa2edd73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:37.443893 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b9c86-71f5-4fd9-ac9f-9efcaa2edd73 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:37.444435 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b9c86-71f5-4fd9-ac9f-9efcaa2edd73 admin admin] PATH_INFO: `/v3/projects/d913701e44d244b9b5c2cda61d85cd32` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:37.445849 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740b9c86-71f5-4fd9-ac9f-9efcaa2edd73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d913701e44d244b9b5c2cda61d85cd32)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:37.447723 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: WARNING py.warnings [None req-740b9c86-71f5-4fd9-ac9f-9efcaa2edd73 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 08:44:37.447723 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: warnings.warn(msg) Oct 15 08:44:37.447723 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: Oct 15 08:44:37.449388 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740b9c86-71f5-4fd9-ac9f-9efcaa2edd73 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:37.486750 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 669/1343] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:44:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 08:44:37.509167 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161618b5-cde5-4ec7-a1a0-a0d0d2ce5e79 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:37.511133 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-161618b5-cde5-4ec7-a1a0-a0d0d2ce5e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:37.583461 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-740b9c86-71f5-4fd9-ac9f-9efcaa2edd73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd913701e44d244b9b5c2cda61d85cd32', 'request_id': 'req-740b9c86-71f5-4fd9-ac9f-9efcaa2edd73'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:37.624360 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-740b9c86-71f5-4fd9-ac9f-9efcaa2edd73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd913701e44d244b9b5c2cda61d85cd32', 'request_id': 'req-740b9c86-71f5-4fd9-ac9f-9efcaa2edd73'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:37.624767 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161618b5-cde5-4ec7-a1a0-a0d0d2ce5e79 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:37.646379 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-740b9c86-71f5-4fd9-ac9f-9efcaa2edd73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd913701e44d244b9b5c2cda61d85cd32', 'request_id': 'req-740b9c86-71f5-4fd9-ac9f-9efcaa2edd73'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:37.648346 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:44:37.682521 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-740b9c86-71f5-4fd9-ac9f-9efcaa2edd73 admin admin] The token cache is being invalidate because project d913701e44d244b9b5c2cda61d85cd32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 08:44:37.682521 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-740b9c86-71f5-4fd9-ac9f-9efcaa2edd73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:37.684837 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-740b9c86-71f5-4fd9-ac9f-9efcaa2edd73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:37.687596 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 675/1344] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 08:44:37 2022] DELETE /identity/v3/projects/d913701e44d244b9b5c2cda61d85cd32 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:44:37.694613 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f68a04-5303-49eb-b9f8-ea7c250b9590 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:37.694999 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f68a04-5303-49eb-b9f8-ea7c250b9590 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:37.695350 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f68a04-5303-49eb-b9f8-ea7c250b9590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:37.732280 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa8329f-4687-497d-a844-d1676b1d9218 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:37.733013 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa8329f-4687-497d-a844-d1676b1d9218 service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:37.733360 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa8329f-4687-497d-a844-d1676b1d9218 service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:37.733719 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa8329f-4687-497d-a844-d1676b1d9218 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:37.735680 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-0aa8329f-4687-497d-a844-d1676b1d9218 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:37.742838 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-c3f68a04-5303-49eb-b9f8-ea7c250b9590 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:37.744958 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-c3f68a04-5303-49eb-b9f8-ea7c250b9590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:37.761064 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:44:37.923542 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 676/1345] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:44:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:37.926835 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeff75f-84f7-4b17-ab75-55186b8ee3b6 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:37.930115 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeff75f-84f7-4b17-ab75-55186b8ee3b6 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:37.931761 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeff75f-84f7-4b17-ab75-55186b8ee3b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:37.941458 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa8329f-4687-497d-a844-d1676b1d9218 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:37.944515 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa8329f-4687-497d-a844-d1676b1d9218 service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:38.003023 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-1eeff75f-84f7-4b17-ab75-55186b8ee3b6 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:38.006461 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-1eeff75f-84f7-4b17-ab75-55186b8ee3b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:38.017260 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 670/1346] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:44:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 08:44:38.030506 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 677/1347] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:44:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:38.034768 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa8329f-4687-497d-a844-d1676b1d9218 service neutron] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:38.069768 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa8329f-4687-497d-a844-d1676b1d9218 service neutron] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:38.088164 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f8f613-19e3-4bc1-8fc1-47e6e78b1926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:38.089651 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f8f613-19e3-4bc1-8fc1-47e6e78b1926 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:38.090312 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f8f613-19e3-4bc1-8fc1-47e6e78b1926 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:38.090312 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f8f613-19e3-4bc1-8fc1-47e6e78b1926 admin admin] PATH_INFO: `/v3/domains` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:38.091730 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f8f613-19e3-4bc1-8fc1-47e6e78b1926 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:44:38.092151 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f8f613-19e3-4bc1-8fc1-47e6e78b1926 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:38.103976 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f8f613-19e3-4bc1-8fc1-47e6e78b1926 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:38.137322 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 671/1348] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 08:44:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:44:38.138967 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeff75f-84f7-4b17-ab75-55186b8ee3b6 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:38.154599 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeff75f-84f7-4b17-ab75-55186b8ee3b6 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:38.172743 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f7aa13-0ba9-4793-9c8a-cb850baf699c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:38.173932 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f7aa13-0ba9-4793-9c8a-cb850baf699c admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:38.174388 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f7aa13-0ba9-4793-9c8a-cb850baf699c admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:38.174854 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f7aa13-0ba9-4793-9c8a-cb850baf699c admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:38.175694 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f7aa13-0ba9-4793-9c8a-cb850baf699c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:38.179284 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f7aa13-0ba9-4793-9c8a-cb850baf699c admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:38.260146 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 678/1349] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 08:44:38 2022] POST /identity/v3/projects => generated 373 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:44:38.264536 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f8f613-19e3-4bc1-8fc1-47e6e78b1926 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:38.278302 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f8f613-19e3-4bc1-8fc1-47e6e78b1926 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:38.374837 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902f8aa6-1608-44c5-b3a8-6795d07c2794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:38.376082 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f8aa6-1608-44c5-b3a8-6795d07c2794 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:38.376596 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f8aa6-1608-44c5-b3a8-6795d07c2794 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:38.376941 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f8aa6-1608-44c5-b3a8-6795d07c2794 admin admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:38.377668 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902f8aa6-1608-44c5-b3a8-6795d07c2794 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:38.379859 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902f8aa6-1608-44c5-b3a8-6795d07c2794 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:38.425311 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 672/1350] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:44:38 2022] POST /identity/v3/users => generated 287 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:44:38.429755 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f7aa13-0ba9-4793-9c8a-cb850baf699c admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:38.450442 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f7aa13-0ba9-4793-9c8a-cb850baf699c admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:38.464171 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241c1a89-4c9a-417f-aef2-8eb8fcf033fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:38.467767 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241c1a89-4c9a-417f-aef2-8eb8fcf033fd admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:38.468392 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241c1a89-4c9a-417f-aef2-8eb8fcf033fd admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:38.468917 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241c1a89-4c9a-417f-aef2-8eb8fcf033fd admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:38.472034 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241c1a89-4c9a-417f-aef2-8eb8fcf033fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:38.478222 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241c1a89-4c9a-417f-aef2-8eb8fcf033fd admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:38.506462 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902f8aa6-1608-44c5-b3a8-6795d07c2794 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:38.506697 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 679/1351] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:44:38 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:44:38.524333 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902f8aa6-1608-44c5-b3a8-6795d07c2794 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:38.537152 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2763dd-5a61-428f-85da-7d8bc76f09b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:38.538108 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2763dd-5a61-428f-85da-7d8bc76f09b5 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:38.539131 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2763dd-5a61-428f-85da-7d8bc76f09b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:38.540594 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2763dd-5a61-428f-85da-7d8bc76f09b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:38.541562 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2763dd-5a61-428f-85da-7d8bc76f09b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:38.548402 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2763dd-5a61-428f-85da-7d8bc76f09b5 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:38.584242 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 673/1352] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:44:38 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:44:38.586905 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241c1a89-4c9a-417f-aef2-8eb8fcf033fd admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:38.601806 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241c1a89-4c9a-417f-aef2-8eb8fcf033fd admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:38.612379 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd8e7d8-a28c-4478-ab4c-18109ffa8c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:38.615070 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd8e7d8-a28c-4478-ab4c-18109ffa8c3a admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:38.615537 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd8e7d8-a28c-4478-ab4c-18109ffa8c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:38.616853 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd8e7d8-a28c-4478-ab4c-18109ffa8c3a admin admin] PATH_INFO: `/v3/projects/7fd58de14fda4ec8bb3c59e0938fac20/users/344016e60c0543a4a2494c4c239287dd/roles/3248f0268738489b883041851d681bb7` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:38.664620 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd8e7d8-a28c-4478-ab4c-18109ffa8c3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fd58de14fda4ec8bb3c59e0938fac20, user_id=344016e60c0543a4a2494c4c239287dd, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:38.670468 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd8e7d8-a28c-4478-ab4c-18109ffa8c3a admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:38.702740 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 680/1353] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:44:38 2022] PUT /identity/v3/projects/7fd58de14fda4ec8bb3c59e0938fac20/users/344016e60c0543a4a2494c4c239287dd/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:44:38.714515 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131df4ca-0773-4bdf-9efb-a540e6a91beb None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:38.715851 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131df4ca-0773-4bdf-9efb-a540e6a91beb None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:38.716311 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131df4ca-0773-4bdf-9efb-a540e6a91beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:38.795386 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-131df4ca-0773-4bdf-9efb-a540e6a91beb None None] MFA Rules not processed for user `344016e60c0543a4a2494c4c239287dd`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:38.798564 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-131df4ca-0773-4bdf-9efb-a540e6a91beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:38.997942 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 674/1354] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:44:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:38.999704 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd8e7d8-a28c-4478-ab4c-18109ffa8c3a admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:39.186965 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131df4ca-0773-4bdf-9efb-a540e6a91beb None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:39.187155 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd8e7d8-a28c-4478-ab4c-18109ffa8c3a admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:39.188533 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-131df4ca-0773-4bdf-9efb-a540e6a91beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:39.210455 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ae4460-fb2a-478f-863f-151d648961e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:39.211101 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae4460-fb2a-478f-863f-151d648961e1 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:39.211391 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae4460-fb2a-478f-863f-151d648961e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:39.212204 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae4460-fb2a-478f-863f-151d648961e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:39.212856 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ae4460-fb2a-478f-863f-151d648961e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:39.214822 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ae4460-fb2a-478f-863f-151d648961e1 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:39.223932 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:44:39.236386 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131df4ca-0773-4bdf-9efb-a540e6a91beb None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:39.251230 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb1ec10-fc9e-428d-a7b7-e1423aea26cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:39.251583 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1ec10-fc9e-428d-a7b7-e1423aea26cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:39.252919 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1ec10-fc9e-428d-a7b7-e1423aea26cb admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:39.253788 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1ec10-fc9e-428d-a7b7-e1423aea26cb admin admin] PATH_INFO: `/v3/users/6cadc5fb6f834c04925b54e48296e668` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:39.265059 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb1ec10-fc9e-428d-a7b7-e1423aea26cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cadc5fb6f834c04925b54e48296e668)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:39.270472 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: WARNING py.warnings [None req-1fb1ec10-fc9e-428d-a7b7-e1423aea26cb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 08:44:39.270472 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: warnings.warn(msg) Oct 15 08:44:39.270472 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: Oct 15 08:44:39.271454 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb1ec10-fc9e-428d-a7b7-e1423aea26cb admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:39.314072 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 681/1355] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 08:44:38 2022] POST /identity/v3/projects => generated 369 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:44:39.319026 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ae4460-fb2a-478f-863f-151d648961e1 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:39.366439 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:44:39.420807 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-1fb1ec10-fc9e-428d-a7b7-e1423aea26cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cadc5fb6f834c04925b54e48296e668'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:39.421876 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-1fb1ec10-fc9e-428d-a7b7-e1423aea26cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cadc5fb6f834c04925b54e48296e668'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:39.465711 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-1fb1ec10-fc9e-428d-a7b7-e1423aea26cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cadc5fb6f834c04925b54e48296e668'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:39.479298 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-1fb1ec10-fc9e-428d-a7b7-e1423aea26cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cadc5fb6f834c04925b54e48296e668'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:39.480343 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-1fb1ec10-fc9e-428d-a7b7-e1423aea26cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cadc5fb6f834c04925b54e48296e668'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:39.517912 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 675/1356] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 08:44:39 2022] DELETE /identity/v3/users/6cadc5fb6f834c04925b54e48296e668 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:44:39.519814 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a8d453-62b3-4f73-bd2b-f3fe412c7afd None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:39.520317 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a8d453-62b3-4f73-bd2b-f3fe412c7afd None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:39.520740 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a8d453-62b3-4f73-bd2b-f3fe412c7afd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:39.531196 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ae4460-fb2a-478f-863f-151d648961e1 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:39.563278 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-01a8d453-62b3-4f73-bd2b-f3fe412c7afd None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:39.565057 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-01a8d453-62b3-4f73-bd2b-f3fe412c7afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:39.651648 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 676/1357] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:44:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:39.666271 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dbec4a-f946-47a8-9d81-33591675ad1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:39.666960 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dbec4a-f946-47a8-9d81-33591675ad1c admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:39.667245 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dbec4a-f946-47a8-9d81-33591675ad1c admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:39.667522 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dbec4a-f946-47a8-9d81-33591675ad1c admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:39.668186 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dbec4a-f946-47a8-9d81-33591675ad1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:39.670184 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dbec4a-f946-47a8-9d81-33591675ad1c admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:39.701871 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 682/1358] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:44:39 2022] POST /identity/v3/users => generated 291 bytes in 384 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:44:39.702898 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a8d453-62b3-4f73-bd2b-f3fe412c7afd None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:39.704420 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-01a8d453-62b3-4f73-bd2b-f3fe412c7afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:39.733717 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a8d453-62b3-4f73-bd2b-f3fe412c7afd None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:39.748129 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776640b1-ba33-4d49-9fd2-5644ab86124d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:39.749104 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776640b1-ba33-4d49-9fd2-5644ab86124d admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:39.749471 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776640b1-ba33-4d49-9fd2-5644ab86124d admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:39.750185 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776640b1-ba33-4d49-9fd2-5644ab86124d admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:39.750693 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776640b1-ba33-4d49-9fd2-5644ab86124d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:39.752553 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776640b1-ba33-4d49-9fd2-5644ab86124d admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:39.777017 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dbec4a-f946-47a8-9d81-33591675ad1c admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:39.778793 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-63dbec4a-f946-47a8-9d81-33591675ad1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:39.779661 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 677/1359] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:44:39 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:44:39.789948 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776640b1-ba33-4d49-9fd2-5644ab86124d admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:39.803204 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776640b1-ba33-4d49-9fd2-5644ab86124d admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:39.824849 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b3760f-65e7-4638-9a83-9b684ad1bff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:39.825549 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b3760f-65e7-4638-9a83-9b684ad1bff9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:39.825870 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b3760f-65e7-4638-9a83-9b684ad1bff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:39.826156 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b3760f-65e7-4638-9a83-9b684ad1bff9 admin admin] PATH_INFO: `/v3/projects/68baf8cd87e6498d93e7ddebe48a700e/users/1fe755f137d34169bc2b28369aa88d1b/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:39.867753 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b3760f-65e7-4638-9a83-9b684ad1bff9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68baf8cd87e6498d93e7ddebe48a700e, user_id=1fe755f137d34169bc2b28369aa88d1b, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:39.871780 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b3760f-65e7-4638-9a83-9b684ad1bff9 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:39.913931 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dbec4a-f946-47a8-9d81-33591675ad1c admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:39.918826 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 678/1360] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:44:39 2022] PUT /identity/v3/projects/68baf8cd87e6498d93e7ddebe48a700e/users/1fe755f137d34169bc2b28369aa88d1b/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:44:39.922811 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b3760f-65e7-4638-9a83-9b684ad1bff9 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:40.014879 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6164782-bc21-463b-88da-708da46ae7fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:40.015641 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6164782-bc21-463b-88da-708da46ae7fd service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:40.015812 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6164782-bc21-463b-88da-708da46ae7fd service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:40.016091 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6164782-bc21-463b-88da-708da46ae7fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:40.023213 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:44:40.049951 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:44:40.062862 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b3760f-65e7-4638-9a83-9b684ad1bff9 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:40.082949 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69e511f-feb5-4b3b-be44-9a3aab4daa9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:40.083593 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e511f-feb5-4b3b-be44-9a3aab4daa9c admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:40.083897 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e511f-feb5-4b3b-be44-9a3aab4daa9c admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:40.084182 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e511f-feb5-4b3b-be44-9a3aab4daa9c admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:40.084824 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69e511f-feb5-4b3b-be44-9a3aab4daa9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:40.087042 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69e511f-feb5-4b3b-be44-9a3aab4daa9c admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:40.132486 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 679/1361] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:44:39 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 08:44:40.134691 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69e511f-feb5-4b3b-be44-9a3aab4daa9c admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:40.178880 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69e511f-feb5-4b3b-be44-9a3aab4daa9c admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:40.179185 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6164782-bc21-463b-88da-708da46ae7fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:40.181376 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6164782-bc21-463b-88da-708da46ae7fd service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:40.185607 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854ab1f7-7868-4cbc-bfba-2a0a3e23b383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:40.186858 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854ab1f7-7868-4cbc-bfba-2a0a3e23b383 admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:40.187150 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854ab1f7-7868-4cbc-bfba-2a0a3e23b383 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:40.187428 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854ab1f7-7868-4cbc-bfba-2a0a3e23b383 admin admin] PATH_INFO: `/v3/projects/68baf8cd87e6498d93e7ddebe48a700e/users/1fe755f137d34169bc2b28369aa88d1b/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:40.189781 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854ab1f7-7868-4cbc-bfba-2a0a3e23b383 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68baf8cd87e6498d93e7ddebe48a700e, user_id=1fe755f137d34169bc2b28369aa88d1b, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:40.193880 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854ab1f7-7868-4cbc-bfba-2a0a3e23b383 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:40.210060 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 683/1362] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:44:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:44:40.216611 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 680/1363] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:44:40 2022] PUT /identity/v3/projects/68baf8cd87e6498d93e7ddebe48a700e/users/1fe755f137d34169bc2b28369aa88d1b/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:44:40.222603 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c53e0f9-dfa1-4165-a022-0c7ecb0c2ce3 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:40.223055 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c53e0f9-dfa1-4165-a022-0c7ecb0c2ce3 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:40.223519 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c53e0f9-dfa1-4165-a022-0c7ecb0c2ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:40.287080 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-3c53e0f9-dfa1-4165-a022-0c7ecb0c2ce3 None None] MFA Rules not processed for user `1fe755f137d34169bc2b28369aa88d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:40.288468 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-3c53e0f9-dfa1-4165-a022-0c7ecb0c2ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:40.447219 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 684/1364] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:44:40 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:40.463476 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854ab1f7-7868-4cbc-bfba-2a0a3e23b383 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:40.468618 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-854ab1f7-7868-4cbc-bfba-2a0a3e23b383 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:40.574610 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854ab1f7-7868-4cbc-bfba-2a0a3e23b383 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:40.578900 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c79d2c-b38f-482b-85a4-ab42787c837e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:40.580138 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c79d2c-b38f-482b-85a4-ab42787c837e service cinder] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:40.580428 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c79d2c-b38f-482b-85a4-ab42787c837e service cinder] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:40.580706 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c79d2c-b38f-482b-85a4-ab42787c837e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:40.595461 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c79d2c-b38f-482b-85a4-ab42787c837e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:40.597706 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c79d2c-b38f-482b-85a4-ab42787c837e service cinder] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:40.611224 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 681/1365] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:44:40 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:44:40.700727 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f12e1c-2486-4b64-a81a-a5d30de73f60 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:40.701398 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f12e1c-2486-4b64-a81a-a5d30de73f60 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:40.701923 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f12e1c-2486-4b64-a81a-a5d30de73f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:40.754757 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c79d2c-b38f-482b-85a4-ab42787c837e service cinder] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:40.774464 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c79d2c-b38f-482b-85a4-ab42787c837e service cinder] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:40.782705 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c918ae0-e9f0-432d-ab2c-6d64b7b3f1fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:40.784287 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c918ae0-e9f0-432d-ab2c-6d64b7b3f1fc service cinder] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:40.784682 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c918ae0-e9f0-432d-ab2c-6d64b7b3f1fc service cinder] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:40.785322 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c918ae0-e9f0-432d-ab2c-6d64b7b3f1fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:40.788006 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-4c918ae0-e9f0-432d-ab2c-6d64b7b3f1fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:40.789008 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-57f12e1c-2486-4b64-a81a-a5d30de73f60 None None] MFA Rules not processed for user `9cc9e2cf11804cc39a7de3c314e46679`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:40.791439 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-57f12e1c-2486-4b64-a81a-a5d30de73f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:40.965072 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c918ae0-e9f0-432d-ab2c-6d64b7b3f1fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:40.969642 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 685/1366] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:44:40 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:40.971028 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c98e409-9778-4758-a126-bf7b6621c970 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:40.971316 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c98e409-9778-4758-a126-bf7b6621c970 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:40.971618 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c98e409-9778-4758-a126-bf7b6621c970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:40.979945 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c918ae0-e9f0-432d-ab2c-6d64b7b3f1fc service cinder] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:41.011609 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-5c98e409-9778-4758-a126-bf7b6621c970 None None] MFA Rules not processed for user `9cc9e2cf11804cc39a7de3c314e46679`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:41.012940 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 682/1367] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:44:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:44:41.015642 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-5c98e409-9778-4758-a126-bf7b6621c970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:41.043066 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 686/1368] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:44:40 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:41.057329 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c918ae0-e9f0-432d-ab2c-6d64b7b3f1fc service cinder] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:41.059023 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-4c918ae0-e9f0-432d-ab2c-6d64b7b3f1fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:41.222152 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:44:41.236629 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c918ae0-e9f0-432d-ab2c-6d64b7b3f1fc service cinder] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:41.251402 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b1d34e-8774-431f-9304-30a364f26ec9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0a8af5cadc4cc1ab73e8eb2a20bb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:41.256855 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b1d34e-8774-431f-9304-30a364f26ec9 service nova] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:41.256855 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b1d34e-8774-431f-9304-30a364f26ec9 service nova] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:41.257341 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b1d34e-8774-431f-9304-30a364f26ec9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:41.279966 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b1d34e-8774-431f-9304-30a364f26ec9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:41.281911 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b1d34e-8774-431f-9304-30a364f26ec9 service nova] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:41.315298 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 683/1369] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:44:41 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:44:41.453420 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0595fa-4cd8-4505-8dd3-40881765d3ad None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:41.456136 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0595fa-4cd8-4505-8dd3-40881765d3ad None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:41.456778 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0595fa-4cd8-4505-8dd3-40881765d3ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:41.540299 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-bb0595fa-4cd8-4505-8dd3-40881765d3ad None None] MFA Rules not processed for user `2b1999640b3c41beafefe80d40bf965c`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:41.541634 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-bb0595fa-4cd8-4505-8dd3-40881765d3ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:41.561504 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:44:41.670054 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 687/1370] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:44:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:41.681904 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7eb5e7-9343-49a6-a787-a702a5414999 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:41.682444 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7eb5e7-9343-49a6-a787-a702a5414999 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:41.682444 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7eb5e7-9343-49a6-a787-a702a5414999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:41.706010 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-7b7eb5e7-9343-49a6-a787-a702a5414999 None None] MFA Rules not processed for user `2b1999640b3c41beafefe80d40bf965c`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:41.707315 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-7b7eb5e7-9343-49a6-a787-a702a5414999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:41.719330 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0595fa-4cd8-4505-8dd3-40881765d3ad None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:41.722044 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-bb0595fa-4cd8-4505-8dd3-40881765d3ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:41.746365 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 684/1371] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:44:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:41.747682 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6479796c-05c8-42c6-aa01-e412ba6a7abb None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:41.747972 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6479796c-05c8-42c6-aa01-e412ba6a7abb None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:41.748269 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6479796c-05c8-42c6-aa01-e412ba6a7abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:41.823773 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-6479796c-05c8-42c6-aa01-e412ba6a7abb None None] MFA Rules not processed for user `388ca829d87348bd9b524fd9a80e578b`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:41.825764 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-6479796c-05c8-42c6-aa01-e412ba6a7abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:41.897911 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0595fa-4cd8-4505-8dd3-40881765d3ad None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:41.911223 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a48d351-3464-4bcf-b7be-5f1acbfd05c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:41.911885 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a48d351-3464-4bcf-b7be-5f1acbfd05c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:41.919676 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a48d351-3464-4bcf-b7be-5f1acbfd05c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:41.919900 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a48d351-3464-4bcf-b7be-5f1acbfd05c3 admin admin] PATH_INFO: `/v3/projects/4f5a931619924d06985b4538a06ff9f9` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:41.921163 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a48d351-3464-4bcf-b7be-5f1acbfd05c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f5a931619924d06985b4538a06ff9f9)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:41.923499 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a48d351-3464-4bcf-b7be-5f1acbfd05c3 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:41.942963 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 685/1372] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:44:41 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:41.944216 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6479796c-05c8-42c6-aa01-e412ba6a7abb None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:41.946568 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-6479796c-05c8-42c6-aa01-e412ba6a7abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:41.967184 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6479796c-05c8-42c6-aa01-e412ba6a7abb None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:41.974059 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7238c7f7-421e-4333-9807-4cb471aac677 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0a8af5cadc4cc1ab73e8eb2a20bb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:41.975064 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7238c7f7-421e-4333-9807-4cb471aac677 service nova] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:41.984553 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7238c7f7-421e-4333-9807-4cb471aac677 service nova] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:41.984997 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7238c7f7-421e-4333-9807-4cb471aac677 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:42.006222 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7238c7f7-421e-4333-9807-4cb471aac677 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:42.008160 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7238c7f7-421e-4333-9807-4cb471aac677 service nova] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:42.020043 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 686/1373] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:44:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:44:42.043649 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-3a48d351-3464-4bcf-b7be-5f1acbfd05c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f5a931619924d06985b4538a06ff9f9', 'request_id': 'req-3a48d351-3464-4bcf-b7be-5f1acbfd05c3'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:42.052204 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-3a48d351-3464-4bcf-b7be-5f1acbfd05c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f5a931619924d06985b4538a06ff9f9', 'request_id': 'req-3a48d351-3464-4bcf-b7be-5f1acbfd05c3'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:42.066074 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7238c7f7-421e-4333-9807-4cb471aac677 service nova] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:42.068176 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-7238c7f7-421e-4333-9807-4cb471aac677 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:42.080573 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-3a48d351-3464-4bcf-b7be-5f1acbfd05c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f5a931619924d06985b4538a06ff9f9', 'request_id': 'req-3a48d351-3464-4bcf-b7be-5f1acbfd05c3'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:42.093114 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7238c7f7-421e-4333-9807-4cb471aac677 service nova] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:42.106140 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-3a48d351-3464-4bcf-b7be-5f1acbfd05c3 admin admin] The token cache is being invalidate because project 4f5a931619924d06985b4538a06ff9f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 08:44:42.106435 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-3a48d351-3464-4bcf-b7be-5f1acbfd05c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:42.107276 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-3a48d351-3464-4bcf-b7be-5f1acbfd05c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:42.111533 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 688/1374] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 08:44:41 2022] DELETE /identity/v3/projects/4f5a931619924d06985b4538a06ff9f9 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 08:44:42.111533 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eefc0b-d7e8-4381-9434-287bf7f74beb None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:42.111533 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eefc0b-d7e8-4381-9434-287bf7f74beb None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:42.111533 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eefc0b-d7e8-4381-9434-287bf7f74beb None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:42.116815 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 689/1375] 10.222.0.30 () {54 vars in 912 bytes} [Sat Oct 15 08:44:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:44:42.118343 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eefc0b-d7e8-4381-9434-287bf7f74beb None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:42.120304 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-d5eefc0b-d7e8-4381-9434-287bf7f74beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:42.217651 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:44:42.276572 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eefc0b-d7e8-4381-9434-287bf7f74beb None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:42.279036 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c81b5a-f711-4569-89b6-ef547f08e250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:42.279990 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c81b5a-f711-4569-89b6-ef547f08e250 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:42.280278 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c81b5a-f711-4569-89b6-ef547f08e250 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:42.280557 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c81b5a-f711-4569-89b6-ef547f08e250 admin admin] PATH_INFO: `/v3/projects/1df92a7d96684bed87c38a18e416c0e4` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:42.282141 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c81b5a-f711-4569-89b6-ef547f08e250 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1df92a7d96684bed87c38a18e416c0e4)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:42.284141 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c81b5a-f711-4569-89b6-ef547f08e250 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:42.290611 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1004a987-91bd-45d3-9dcc-a9adad034feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:42.291372 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1004a987-91bd-45d3-9dcc-a9adad034feb admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:42.291537 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1004a987-91bd-45d3-9dcc-a9adad034feb admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:42.291815 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1004a987-91bd-45d3-9dcc-a9adad034feb admin admin] PATH_INFO: `/v3/users/ba3556a1334f46369dbaac93f52a8ff4` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:42.294156 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1004a987-91bd-45d3-9dcc-a9adad034feb admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba3556a1334f46369dbaac93f52a8ff4)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:42.301447 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1004a987-91bd-45d3-9dcc-a9adad034feb admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:42.412838 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-78c81b5a-f711-4569-89b6-ef547f08e250 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1df92a7d96684bed87c38a18e416c0e4', 'request_id': 'req-78c81b5a-f711-4569-89b6-ef547f08e250'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:42.424893 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-1004a987-91bd-45d3-9dcc-a9adad034feb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba3556a1334f46369dbaac93f52a8ff4'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:42.427049 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-1004a987-91bd-45d3-9dcc-a9adad034feb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba3556a1334f46369dbaac93f52a8ff4'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:42.431196 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-78c81b5a-f711-4569-89b6-ef547f08e250 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1df92a7d96684bed87c38a18e416c0e4', 'request_id': 'req-78c81b5a-f711-4569-89b6-ef547f08e250'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:42.441231 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-1004a987-91bd-45d3-9dcc-a9adad034feb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba3556a1334f46369dbaac93f52a8ff4'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:42.450475 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-78c81b5a-f711-4569-89b6-ef547f08e250 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1df92a7d96684bed87c38a18e416c0e4', 'request_id': 'req-78c81b5a-f711-4569-89b6-ef547f08e250'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:42.454055 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:44:42.478631 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-78c81b5a-f711-4569-89b6-ef547f08e250 admin admin] The token cache is being invalidate because project 1df92a7d96684bed87c38a18e416c0e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 08:44:42.479510 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-78c81b5a-f711-4569-89b6-ef547f08e250 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:42.483600 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-78c81b5a-f711-4569-89b6-ef547f08e250 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:42.484690 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-1004a987-91bd-45d3-9dcc-a9adad034feb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba3556a1334f46369dbaac93f52a8ff4'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:42.486118 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 687/1376] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 08:44:42 2022] DELETE /identity/v3/projects/1df92a7d96684bed87c38a18e416c0e4 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:44:42.490771 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a67c38-1e12-4dfc-8a73-17e9b2046b8e None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:42.491171 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a67c38-1e12-4dfc-8a73-17e9b2046b8e None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:42.491645 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a67c38-1e12-4dfc-8a73-17e9b2046b8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:42.498627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-1004a987-91bd-45d3-9dcc-a9adad034feb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba3556a1334f46369dbaac93f52a8ff4'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:42.510573 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 690/1377] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 08:44:42 2022] DELETE /identity/v3/users/ba3556a1334f46369dbaac93f52a8ff4 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:44:42.511957 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13831c7-b3e6-4859-b84f-98d38b6494f2 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:42.512248 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13831c7-b3e6-4859-b84f-98d38b6494f2 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:42.512530 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13831c7-b3e6-4859-b84f-98d38b6494f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:42.551023 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-a13831c7-b3e6-4859-b84f-98d38b6494f2 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:42.562767 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-a13831c7-b3e6-4859-b84f-98d38b6494f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:42.604912 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-f8a67c38-1e12-4dfc-8a73-17e9b2046b8e None None] MFA Rules not processed for user `8b11e349e4cb4558a4c75d843effc0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:42.607826 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-f8a67c38-1e12-4dfc-8a73-17e9b2046b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:42.709338 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 691/1378] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:44:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:42.711145 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c20fcb-773f-4eab-98bc-119684b7dcdd None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:42.711489 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c20fcb-773f-4eab-98bc-119684b7dcdd None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:42.711774 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c20fcb-773f-4eab-98bc-119684b7dcdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:42.742376 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 688/1379] 10.222.0.30 () {58 vars in 1009 bytes} [Sat Oct 15 08:44:42 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:42.762995 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-b8c20fcb-773f-4eab-98bc-119684b7dcdd None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:42.764253 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-b8c20fcb-773f-4eab-98bc-119684b7dcdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:42.790999 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 692/1380] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:44:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:42.792242 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a67c38-1e12-4dfc-8a73-17e9b2046b8e None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:42.808236 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a67c38-1e12-4dfc-8a73-17e9b2046b8e None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:42.823899 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569513a6-8974-4265-9dca-176a39ed9b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:42.824908 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569513a6-8974-4265-9dca-176a39ed9b26 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:42.825391 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569513a6-8974-4265-9dca-176a39ed9b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:42.825870 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569513a6-8974-4265-9dca-176a39ed9b26 admin admin] PATH_INFO: `/v3/domains` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:42.826751 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569513a6-8974-4265-9dca-176a39ed9b26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:44:42.827241 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569513a6-8974-4265-9dca-176a39ed9b26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:42.829232 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569513a6-8974-4265-9dca-176a39ed9b26 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:42.862321 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 689/1381] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 08:44:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:44:42.863202 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c20fcb-773f-4eab-98bc-119684b7dcdd None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:42.869421 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd8690-bbed-4b5b-96da-f00c6f6da5f0 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:42.869752 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd8690-bbed-4b5b-96da-f00c6f6da5f0 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:42.870039 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd8690-bbed-4b5b-96da-f00c6f6da5f0 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:42.871394 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 690/1382] 10.222.0.30 () {54 vars in 912 bytes} [Sat Oct 15 08:44:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:44:42.873554 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacd8690-bbed-4b5b-96da-f00c6f6da5f0 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:42.875294 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c20fcb-773f-4eab-98bc-119684b7dcdd None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:42.879613 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-eacd8690-bbed-4b5b-96da-f00c6f6da5f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:42.887747 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a3c4d-28a6-4b6c-a64c-3e345bbd75b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:42.888679 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a3c4d-28a6-4b6c-a64c-3e345bbd75b3 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:42.889012 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a3c4d-28a6-4b6c-a64c-3e345bbd75b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:42.889787 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a3c4d-28a6-4b6c-a64c-3e345bbd75b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:42.891112 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298a3c4d-28a6-4b6c-a64c-3e345bbd75b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:42.897615 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298a3c4d-28a6-4b6c-a64c-3e345bbd75b3 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:42.904107 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacd8690-bbed-4b5b-96da-f00c6f6da5f0 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:42.910249 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faa7b69-0df9-456f-b19a-b37c3f8afea6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:42.911366 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa7b69-0df9-456f-b19a-b37c3f8afea6 service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:42.911366 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa7b69-0df9-456f-b19a-b37c3f8afea6 service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:42.911546 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa7b69-0df9-456f-b19a-b37c3f8afea6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:42.926115 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faa7b69-0df9-456f-b19a-b37c3f8afea6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:42.928036 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faa7b69-0df9-456f-b19a-b37c3f8afea6 service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:42.947151 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 691/1383] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:44:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:44:42.948533 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597351d5-89ad-41ea-abf9-825774721105 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:42.949190 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597351d5-89ad-41ea-abf9-825774721105 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:42.949190 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597351d5-89ad-41ea-abf9-825774721105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:42.974057 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-597351d5-89ad-41ea-abf9-825774721105 None None] MFA Rules not processed for user `8b11e349e4cb4558a4c75d843effc0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:42.975460 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-597351d5-89ad-41ea-abf9-825774721105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:42.998529 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 693/1384] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 08:44:42 2022] POST /identity/v3/projects => generated 373 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:44:43.000365 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 692/1385] 10.222.0.30 () {58 vars in 1009 bytes} [Sat Oct 15 08:44:42 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:43.011991 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a3c4d-28a6-4b6c-a64c-3e345bbd75b3 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:43.061180 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597351d5-89ad-41ea-abf9-825774721105 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:43.073283 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597351d5-89ad-41ea-abf9-825774721105 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:43.156836 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:44:43.242091 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22be69d1-e9a7-46fb-8847-68a795dceedb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:43.243259 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be69d1-e9a7-46fb-8847-68a795dceedb service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:43.243695 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be69d1-e9a7-46fb-8847-68a795dceedb service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:43.243982 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be69d1-e9a7-46fb-8847-68a795dceedb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:43.265704 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22be69d1-e9a7-46fb-8847-68a795dceedb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:43.265704 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22be69d1-e9a7-46fb-8847-68a795dceedb service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:43.279992 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a3c4d-28a6-4b6c-a64c-3e345bbd75b3 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:43.288499 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 693/1386] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:44:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:44:43.296044 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae3effe-8a47-4ab3-8fea-02f55ca71727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:43.297599 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae3effe-8a47-4ab3-8fea-02f55ca71727 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:43.299792 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae3effe-8a47-4ab3-8fea-02f55ca71727 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:43.300232 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae3effe-8a47-4ab3-8fea-02f55ca71727 admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:43.302754 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae3effe-8a47-4ab3-8fea-02f55ca71727 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:43.320330 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae3effe-8a47-4ab3-8fea-02f55ca71727 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:43.402819 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 694/1387] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:44:43 2022] POST /identity/v3/users => generated 287 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:44:43.404263 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22be69d1-e9a7-46fb-8847-68a795dceedb service neutron] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:43.420755 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22be69d1-e9a7-46fb-8847-68a795dceedb service neutron] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:43.426609 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daa28a3-4731-4f2c-8da9-80dad78a51be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:43.427238 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daa28a3-4731-4f2c-8da9-80dad78a51be admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:43.427520 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daa28a3-4731-4f2c-8da9-80dad78a51be admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:43.427812 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daa28a3-4731-4f2c-8da9-80dad78a51be admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:43.428501 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daa28a3-4731-4f2c-8da9-80dad78a51be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:43.438617 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daa28a3-4731-4f2c-8da9-80dad78a51be admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:43.456870 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 694/1388] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:44:43 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:44:43.458259 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae3effe-8a47-4ab3-8fea-02f55ca71727 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:43.482402 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae3effe-8a47-4ab3-8fea-02f55ca71727 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:43.527677 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b42675-efbe-40b2-86bf-736a4cd78830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:43.528439 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b42675-efbe-40b2-86bf-736a4cd78830 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:43.528648 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b42675-efbe-40b2-86bf-736a4cd78830 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:43.528925 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b42675-efbe-40b2-86bf-736a4cd78830 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:43.529964 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b42675-efbe-40b2-86bf-736a4cd78830 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:43.532550 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b42675-efbe-40b2-86bf-736a4cd78830 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:43.564298 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 695/1389] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:44:43 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:44:43.565634 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daa28a3-4731-4f2c-8da9-80dad78a51be admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:43.589657 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daa28a3-4731-4f2c-8da9-80dad78a51be admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:43.605349 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e14a93-71c0-4deb-8536-14e3a1c49823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:43.606241 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e14a93-71c0-4deb-8536-14e3a1c49823 admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:43.606626 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e14a93-71c0-4deb-8536-14e3a1c49823 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:43.607013 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e14a93-71c0-4deb-8536-14e3a1c49823 admin admin] PATH_INFO: `/v3/projects/d18c41fc79fe4fd2be58ca56ba77a660/users/2fe5480da9a34418a49c657c90542988/roles/3248f0268738489b883041851d681bb7` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:43.646336 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e14a93-71c0-4deb-8536-14e3a1c49823 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d18c41fc79fe4fd2be58ca56ba77a660, user_id=2fe5480da9a34418a49c657c90542988, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:43.650072 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e14a93-71c0-4deb-8536-14e3a1c49823 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:43.693415 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 695/1390] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:44:43 2022] PUT /identity/v3/projects/d18c41fc79fe4fd2be58ca56ba77a660/users/2fe5480da9a34418a49c657c90542988/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:44:43.694903 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd6e8ca-5ba3-4791-afd8-9d29621e5eb4 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:43.695195 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd6e8ca-5ba3-4791-afd8-9d29621e5eb4 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:43.695548 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd6e8ca-5ba3-4791-afd8-9d29621e5eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:43.761876 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-9fd6e8ca-5ba3-4791-afd8-9d29621e5eb4 None None] MFA Rules not processed for user `2fe5480da9a34418a49c657c90542988`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:43.763138 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-9fd6e8ca-5ba3-4791-afd8-9d29621e5eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:43.924462 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 696/1391] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:44:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:43.956165 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3172124c-93fe-4efc-bb7e-8c3a0573352c None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:43.957021 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3172124c-93fe-4efc-bb7e-8c3a0573352c None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:43.957466 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3172124c-93fe-4efc-bb7e-8c3a0573352c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:43.970756 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd6e8ca-5ba3-4791-afd8-9d29621e5eb4 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:43.972636 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-9fd6e8ca-5ba3-4791-afd8-9d29621e5eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:44.041468 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-3172124c-93fe-4efc-bb7e-8c3a0573352c None None] MFA Rules not processed for user `a5d85f7efca040cc81253dfbf0de7c42`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:44.042748 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-3172124c-93fe-4efc-bb7e-8c3a0573352c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:44.090606 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:44:44.190867 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 696/1392] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:44:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:44.192180 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3172124c-93fe-4efc-bb7e-8c3a0573352c None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:44.198608 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-3172124c-93fe-4efc-bb7e-8c3a0573352c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:44.207414 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd6e8ca-5ba3-4791-afd8-9d29621e5eb4 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:44.220907 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02aae7c-8028-4ae5-a683-ac1547c41f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:44.221247 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02aae7c-8028-4ae5-a683-ac1547c41f1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:44.221538 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02aae7c-8028-4ae5-a683-ac1547c41f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:44.221845 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02aae7c-8028-4ae5-a683-ac1547c41f1b admin admin] PATH_INFO: `/v3/projects/a1a3585c814a4050996be4a854796598` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:44.223260 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02aae7c-8028-4ae5-a683-ac1547c41f1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1a3585c814a4050996be4a854796598)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:44.225185 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02aae7c-8028-4ae5-a683-ac1547c41f1b admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:44.407683 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:44:44.422078 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d02aae7c-8028-4ae5-a683-ac1547c41f1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1a3585c814a4050996be4a854796598', 'request_id': 'req-d02aae7c-8028-4ae5-a683-ac1547c41f1b'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:44.441463 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d02aae7c-8028-4ae5-a683-ac1547c41f1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1a3585c814a4050996be4a854796598', 'request_id': 'req-d02aae7c-8028-4ae5-a683-ac1547c41f1b'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:44.463401 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d02aae7c-8028-4ae5-a683-ac1547c41f1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1a3585c814a4050996be4a854796598', 'request_id': 'req-d02aae7c-8028-4ae5-a683-ac1547c41f1b'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:44.466063 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3172124c-93fe-4efc-bb7e-8c3a0573352c None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:44.504355 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d02aae7c-8028-4ae5-a683-ac1547c41f1b admin admin] The token cache is being invalidate because project a1a3585c814a4050996be4a854796598 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 08:44:44.505543 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d02aae7c-8028-4ae5-a683-ac1547c41f1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:44.507590 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d02aae7c-8028-4ae5-a683-ac1547c41f1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:44.511670 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 697/1393] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 08:44:43 2022] DELETE /identity/v3/projects/a1a3585c814a4050996be4a854796598 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:44:44.519446 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02aae7c-8028-4ae5-a683-ac1547c41f1b admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:44.525271 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-d02aae7c-8028-4ae5-a683-ac1547c41f1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:44.622810 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b6b26-7cfc-47f8-bdfa-28dd25a51069 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0a8af5cadc4cc1ab73e8eb2a20bb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:44.623528 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b6b26-7cfc-47f8-bdfa-28dd25a51069 service nova] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:44.623914 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b6b26-7cfc-47f8-bdfa-28dd25a51069 service nova] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:44.624289 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b6b26-7cfc-47f8-bdfa-28dd25a51069 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:44.626489 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-0e9b6b26-7cfc-47f8-bdfa-28dd25a51069 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:44.708146 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02aae7c-8028-4ae5-a683-ac1547c41f1b admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:44.716852 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d832766-071f-481f-b620-84472e347adf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38986ca2612c4fc59d960cd693834f9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:44.718204 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9b6b26-7cfc-47f8-bdfa-28dd25a51069 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:44.718576 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d832766-071f-481f-b620-84472e347adf service placement] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:44.719033 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d832766-071f-481f-b620-84472e347adf service placement] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:44.719484 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d832766-071f-481f-b620-84472e347adf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:44.720288 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9b6b26-7cfc-47f8-bdfa-28dd25a51069 service nova] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:44.722335 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-7d832766-071f-481f-b620-84472e347adf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:44.736393 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 697/1394] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:44:44 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:44:44.737256 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b6b26-7cfc-47f8-bdfa-28dd25a51069 service nova] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:44.744671 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-0e9b6b26-7cfc-47f8-bdfa-28dd25a51069 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:44.754828 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:44:44.755254 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d832766-071f-481f-b620-84472e347adf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:44.757058 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d832766-071f-481f-b620-84472e347adf service placement] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:44.769641 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b6b26-7cfc-47f8-bdfa-28dd25a51069 service nova] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:44.780949 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 698/1395] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:44:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:44:44.782367 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c64ba9-2a34-468c-8afb-57607b3ae0de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0a8af5cadc4cc1ab73e8eb2a20bb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:44.782974 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c64ba9-2a34-468c-8afb-57607b3ae0de service nova] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:44.783901 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf34f9c-ef14-4349-94ef-3dd38e59a39a None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:44.784242 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf34f9c-ef14-4349-94ef-3dd38e59a39a None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:44.784975 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf34f9c-ef14-4349-94ef-3dd38e59a39a None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:44.785562 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c64ba9-2a34-468c-8afb-57607b3ae0de service nova] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:44.786006 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c64ba9-2a34-468c-8afb-57607b3ae0de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:44.793153 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 699/1396] 10.222.0.30 () {54 vars in 961 bytes} [Sat Oct 15 08:44:44 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:44:44.795471 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16199429-84b6-4eb9-943b-26901fc79373 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:44.795471 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16199429-84b6-4eb9-943b-26901fc79373 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:44.795471 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16199429-84b6-4eb9-943b-26901fc79373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:44.796232 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-d1c64ba9-2a34-468c-8afb-57607b3ae0de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:44.838597 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-16199429-84b6-4eb9-943b-26901fc79373 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:44.839826 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-16199429-84b6-4eb9-943b-26901fc79373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:44.953536 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c64ba9-2a34-468c-8afb-57607b3ae0de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:44.957073 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c64ba9-2a34-468c-8afb-57607b3ae0de service nova] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:44.980569 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 698/1397] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:44:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:44:44.982010 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d319a649-b3e3-4b67-a6cc-5f1f66b332ef None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:44.982293 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d319a649-b3e3-4b67-a6cc-5f1f66b332ef None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:44.982568 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d319a649-b3e3-4b67-a6cc-5f1f66b332ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:45.008131 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 700/1398] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:44:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:45.012207 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c20076-cbf9-448c-b704-dde4c4e0f412 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:45.013802 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c20076-cbf9-448c-b704-dde4c4e0f412 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:45.014107 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c20076-cbf9-448c-b704-dde4c4e0f412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:45.079913 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-49c20076-cbf9-448c-b704-dde4c4e0f412 None None] MFA Rules not processed for user `38986ca2612c4fc59d960cd693834f9b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:45.081181 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-49c20076-cbf9-448c-b704-dde4c4e0f412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:45.084578 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-d319a649-b3e3-4b67-a6cc-5f1f66b332ef None None] MFA Rules not processed for user `060d1fae9b0b488da1f7553cabd97d28`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:45.085876 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-d319a649-b3e3-4b67-a6cc-5f1f66b332ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:45.131941 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 701/1399] 10.222.0.30 () {58 vars in 1058 bytes} [Sat Oct 15 08:44:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:45.153348 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40d1511-0fc7-4922-9f19-3f2bfdebd100 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:45.154134 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40d1511-0fc7-4922-9f19-3f2bfdebd100 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:45.154134 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40d1511-0fc7-4922-9f19-3f2bfdebd100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:45.252602 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-e40d1511-0fc7-4922-9f19-3f2bfdebd100 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:45.253929 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-e40d1511-0fc7-4922-9f19-3f2bfdebd100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:45.293419 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 702/1400] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:44:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:45.294468 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40d1511-0fc7-4922-9f19-3f2bfdebd100 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:45.354982 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40d1511-0fc7-4922-9f19-3f2bfdebd100 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:45.374312 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 699/1401] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:44:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 393 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:45.375196 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d319a649-b3e3-4b67-a6cc-5f1f66b332ef None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:45.403684 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1c1967-c9f1-4b6e-b700-6c658cb2e38f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38986ca2612c4fc59d960cd693834f9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:45.404337 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1c1967-c9f1-4b6e-b700-6c658cb2e38f service placement] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:45.404631 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1c1967-c9f1-4b6e-b700-6c658cb2e38f service placement] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:45.404911 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1c1967-c9f1-4b6e-b700-6c658cb2e38f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:45.407392 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d319a649-b3e3-4b67-a6cc-5f1f66b332ef None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:45.420781 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82b8310-90fe-4ebf-8c00-dbfb07f75141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:45.421763 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82b8310-90fe-4ebf-8c00-dbfb07f75141 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:45.422361 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82b8310-90fe-4ebf-8c00-dbfb07f75141 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:45.422825 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82b8310-90fe-4ebf-8c00-dbfb07f75141 admin admin] PATH_INFO: `/v3/domains` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:45.423643 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82b8310-90fe-4ebf-8c00-dbfb07f75141 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:44:45.424124 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82b8310-90fe-4ebf-8c00-dbfb07f75141 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:45.426206 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82b8310-90fe-4ebf-8c00-dbfb07f75141 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:45.438777 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1c1967-c9f1-4b6e-b700-6c658cb2e38f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:45.440424 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1c1967-c9f1-4b6e-b700-6c658cb2e38f service placement] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:45.452234 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 700/1402] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 08:44:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:44:45.459483 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb7b29f-0051-408a-a5d0-1c04cff9fbc5 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:45.460971 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb7b29f-0051-408a-a5d0-1c04cff9fbc5 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:45.462977 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb7b29f-0051-408a-a5d0-1c04cff9fbc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:45.473090 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 703/1403] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:44:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:44:45.485308 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1c1967-c9f1-4b6e-b700-6c658cb2e38f service placement] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:45.530842 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1c1967-c9f1-4b6e-b700-6c658cb2e38f service placement] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:45.547539 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20ff419-e6b6-4b58-8ad3-741dc3e15f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:45.548665 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ff419-e6b6-4b58-8ad3-741dc3e15f20 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:45.548955 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ff419-e6b6-4b58-8ad3-741dc3e15f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:45.549230 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ff419-e6b6-4b58-8ad3-741dc3e15f20 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:45.550047 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20ff419-e6b6-4b58-8ad3-741dc3e15f20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:45.552120 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20ff419-e6b6-4b58-8ad3-741dc3e15f20 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:45.558157 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-ebb7b29f-0051-408a-a5d0-1c04cff9fbc5 None None] MFA Rules not processed for user `9cc9e2cf11804cc39a7de3c314e46679`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:45.559415 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-ebb7b29f-0051-408a-a5d0-1c04cff9fbc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:45.664991 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 704/1404] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 08:44:45 2022] POST /identity/v3/projects => generated 357 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:44:45.665992 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20ff419-e6b6-4b58-8ad3-741dc3e15f20 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:45.729327 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:44:45.794077 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 701/1405] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:44:45 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:45.795394 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb7b29f-0051-408a-a5d0-1c04cff9fbc5 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:45.797317 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-ebb7b29f-0051-408a-a5d0-1c04cff9fbc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:45.826672 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb7b29f-0051-408a-a5d0-1c04cff9fbc5 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:45.904814 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:44:45.906778 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20ff419-e6b6-4b58-8ad3-741dc3e15f20 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:45.927922 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f7465f-b3cc-419f-a4a1-36a4cdb660fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:45.928629 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f7465f-b3cc-419f-a4a1-36a4cdb660fa admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:45.928868 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f7465f-b3cc-419f-a4a1-36a4cdb660fa admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:45.929107 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f7465f-b3cc-419f-a4a1-36a4cdb660fa admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:45.929785 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f7465f-b3cc-419f-a4a1-36a4cdb660fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:45.938062 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f7465f-b3cc-419f-a4a1-36a4cdb660fa admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:46.029070 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 705/1406] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:44:45 2022] POST /identity/v3/users => generated 279 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:44:46.030405 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f7465f-b3cc-419f-a4a1-36a4cdb660fa admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:46.074890 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f7465f-b3cc-419f-a4a1-36a4cdb660fa admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:46.100715 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7c1d55-28be-4ceb-82c1-933aa07b13b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:46.101356 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c1d55-28be-4ceb-82c1-933aa07b13b4 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:46.101641 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c1d55-28be-4ceb-82c1-933aa07b13b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:46.107756 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c1d55-28be-4ceb-82c1-933aa07b13b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:46.108463 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7c1d55-28be-4ceb-82c1-933aa07b13b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:46.115864 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b38f903-29cd-43f7-97d4-df387b901953 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0a8af5cadc4cc1ab73e8eb2a20bb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:46.116518 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b38f903-29cd-43f7-97d4-df387b901953 service nova] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:46.116795 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b38f903-29cd-43f7-97d4-df387b901953 service nova] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:46.117065 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b38f903-29cd-43f7-97d4-df387b901953 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:46.120077 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7c1d55-28be-4ceb-82c1-933aa07b13b4 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:46.167185 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 706/1407] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:44:46 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:44:46.168172 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7c1d55-28be-4ceb-82c1-933aa07b13b4 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:46.205568 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7c1d55-28be-4ceb-82c1-933aa07b13b4 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:46.246115 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb07445-27ea-40da-bc63-e9603ec21f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:46.246752 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb07445-27ea-40da-bc63-e9603ec21f6b admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:46.247049 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb07445-27ea-40da-bc63-e9603ec21f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:46.247317 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb07445-27ea-40da-bc63-e9603ec21f6b admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:46.247970 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb07445-27ea-40da-bc63-e9603ec21f6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:46.256065 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb07445-27ea-40da-bc63-e9603ec21f6b admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:46.319689 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 707/1408] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:44:46 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:44:46.338931 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb07445-27ea-40da-bc63-e9603ec21f6b admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:46.373989 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb07445-27ea-40da-bc63-e9603ec21f6b admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:46.408485 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b38f903-29cd-43f7-97d4-df387b901953 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:46.411077 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b38f903-29cd-43f7-97d4-df387b901953 service nova] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:46.411886 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7b1892-4520-4b30-88e6-566ffd25bda9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:46.412732 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b1892-4520-4b30-88e6-566ffd25bda9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:46.415803 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b1892-4520-4b30-88e6-566ffd25bda9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:46.416276 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b1892-4520-4b30-88e6-566ffd25bda9 admin admin] PATH_INFO: `/v3/projects/24068ef2565945b0bc34a801c9e2b3d8/users/657a8bdd2b834ba9b53c0d8bf47a9e19/roles/3248f0268738489b883041851d681bb7` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:46.457561 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 702/1409] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:44:45 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 663 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:44:46.470938 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7b1892-4520-4b30-88e6-566ffd25bda9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24068ef2565945b0bc34a801c9e2b3d8, user_id=657a8bdd2b834ba9b53c0d8bf47a9e19, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:46.474209 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7b1892-4520-4b30-88e6-566ffd25bda9 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:46.528086 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 708/1410] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:44:46 2022] PUT /identity/v3/projects/24068ef2565945b0bc34a801c9e2b3d8/users/657a8bdd2b834ba9b53c0d8bf47a9e19/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:44:46.539865 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e55a5-9f06-410f-a62c-da037fc19649 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:46.540319 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e55a5-9f06-410f-a62c-da037fc19649 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:46.540736 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e55a5-9f06-410f-a62c-da037fc19649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:46.642256 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-a90e55a5-9f06-410f-a62c-da037fc19649 None None] MFA Rules not processed for user `657a8bdd2b834ba9b53c0d8bf47a9e19`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:46.646367 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-a90e55a5-9f06-410f-a62c-da037fc19649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:46.912383 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7b1892-4520-4b30-88e6-566ffd25bda9 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:46.914937 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 703/1411] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:44:46 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:47.104608 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7b1892-4520-4b30-88e6-566ffd25bda9 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:47.119911 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de78b08a-f7fc-41c6-8f59-3289352258f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:47.121781 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de78b08a-f7fc-41c6-8f59-3289352258f7 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:47.122296 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de78b08a-f7fc-41c6-8f59-3289352258f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:47.122755 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de78b08a-f7fc-41c6-8f59-3289352258f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:47.123664 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de78b08a-f7fc-41c6-8f59-3289352258f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:47.127061 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de78b08a-f7fc-41c6-8f59-3289352258f7 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:47.194581 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 709/1412] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 08:44:46 2022] POST /identity/v3/projects => generated 355 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 08:44:47.198448 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90e55a5-9f06-410f-a62c-da037fc19649 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:47.210642 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90e55a5-9f06-410f-a62c-da037fc19649 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:47.332015 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:44:47.354423 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7607b488-a664-491e-bb13-4d554240ac8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:47.356153 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7607b488-a664-491e-bb13-4d554240ac8b admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:47.356770 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7607b488-a664-491e-bb13-4d554240ac8b admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:47.357190 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7607b488-a664-491e-bb13-4d554240ac8b admin admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:47.358030 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7607b488-a664-491e-bb13-4d554240ac8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:47.360341 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7607b488-a664-491e-bb13-4d554240ac8b admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:47.422461 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 704/1413] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:44:47 2022] POST /identity/v3/users => generated 284 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:44:47.426447 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de78b08a-f7fc-41c6-8f59-3289352258f7 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:47.441734 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de78b08a-f7fc-41c6-8f59-3289352258f7 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:47.456094 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589671f5-2e27-43c9-bd23-698d2b7ef4c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:47.460649 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589671f5-2e27-43c9-bd23-698d2b7ef4c6 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:47.462182 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589671f5-2e27-43c9-bd23-698d2b7ef4c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:47.462629 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589671f5-2e27-43c9-bd23-698d2b7ef4c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:47.464620 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589671f5-2e27-43c9-bd23-698d2b7ef4c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:47.470647 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589671f5-2e27-43c9-bd23-698d2b7ef4c6 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:47.505824 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 710/1414] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:44:47 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:44:47.507491 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7607b488-a664-491e-bb13-4d554240ac8b admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:47.536590 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7607b488-a664-491e-bb13-4d554240ac8b admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:47.545195 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b33893-d23d-49ff-8de4-4d4ee0e22d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:47.546135 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b33893-d23d-49ff-8de4-4d4ee0e22d33 admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:47.546571 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b33893-d23d-49ff-8de4-4d4ee0e22d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:47.548059 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b33893-d23d-49ff-8de4-4d4ee0e22d33 admin admin] PATH_INFO: `/v3/projects/43b43c8c7421405083709d60ab899424/users/c6c9025530064170b7b18c0cc2e0dca2/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:47.584199 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b33893-d23d-49ff-8de4-4d4ee0e22d33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43b43c8c7421405083709d60ab899424, user_id=c6c9025530064170b7b18c0cc2e0dca2, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:47.587502 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b33893-d23d-49ff-8de4-4d4ee0e22d33 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:47.625928 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 705/1415] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:44:47 2022] PUT /identity/v3/projects/43b43c8c7421405083709d60ab899424/users/c6c9025530064170b7b18c0cc2e0dca2/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:44:47.629631 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589671f5-2e27-43c9-bd23-698d2b7ef4c6 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:47.640872 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589671f5-2e27-43c9-bd23-698d2b7ef4c6 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:47.713202 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:44:47.777501 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98feaa28-67d0-4c31-b806-5f3042148297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:47.778206 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98feaa28-67d0-4c31-b806-5f3042148297 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:47.778488 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98feaa28-67d0-4c31-b806-5f3042148297 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:47.778760 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98feaa28-67d0-4c31-b806-5f3042148297 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:47.780571 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98feaa28-67d0-4c31-b806-5f3042148297 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:47.782303 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98feaa28-67d0-4c31-b806-5f3042148297 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:47.813180 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 711/1416] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:44:47 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:44:47.813776 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b33893-d23d-49ff-8de4-4d4ee0e22d33 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:47.829881 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b33893-d23d-49ff-8de4-4d4ee0e22d33 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:47.841346 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b41f146-e2f1-4808-8966-f7c10364f1ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:47.847582 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b41f146-e2f1-4808-8966-f7c10364f1ff admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:47.847918 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b41f146-e2f1-4808-8966-f7c10364f1ff admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:47.849567 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b41f146-e2f1-4808-8966-f7c10364f1ff admin admin] PATH_INFO: `/v3/projects/43b43c8c7421405083709d60ab899424/users/c6c9025530064170b7b18c0cc2e0dca2/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:47.859948 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b41f146-e2f1-4808-8966-f7c10364f1ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=43b43c8c7421405083709d60ab899424, user_id=c6c9025530064170b7b18c0cc2e0dca2, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:47.871769 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b41f146-e2f1-4808-8966-f7c10364f1ff admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:47.927265 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 706/1417] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:44:47 2022] PUT /identity/v3/projects/43b43c8c7421405083709d60ab899424/users/c6c9025530064170b7b18c0cc2e0dca2/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 08:44:47.931283 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7c69c-bb69-4037-821e-4d247de33caa None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:47.932107 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7c69c-bb69-4037-821e-4d247de33caa None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:47.932543 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7c69c-bb69-4037-821e-4d247de33caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:47.990219 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-5ab7c69c-bb69-4037-821e-4d247de33caa None None] MFA Rules not processed for user `c6c9025530064170b7b18c0cc2e0dca2`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:47.991575 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-5ab7c69c-bb69-4037-821e-4d247de33caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:48.120693 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 712/1418] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:44:47 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:48.141903 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b41f146-e2f1-4808-8966-f7c10364f1ff admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:48.144207 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-9b41f146-e2f1-4808-8966-f7c10364f1ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:48.220275 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b41f146-e2f1-4808-8966-f7c10364f1ff admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:48.226295 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eea4f5-71ca-4ca1-a4de-f7d2e19394d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:48.241410 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eea4f5-71ca-4ca1-a4de-f7d2e19394d4 service cinder] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:48.251145 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eea4f5-71ca-4ca1-a4de-f7d2e19394d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:48.252596 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eea4f5-71ca-4ca1-a4de-f7d2e19394d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:48.316900 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eea4f5-71ca-4ca1-a4de-f7d2e19394d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:48.318426 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eea4f5-71ca-4ca1-a4de-f7d2e19394d4 service cinder] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:48.353967 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 707/1419] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:44:48 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:44:48.578448 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab7c69c-bb69-4037-821e-4d247de33caa None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:48.579914 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-5ab7c69c-bb69-4037-821e-4d247de33caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:48.606988 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab7c69c-bb69-4037-821e-4d247de33caa None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:48.622856 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff966de-ea5e-4680-b273-5b59d9fddd88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:48.623515 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff966de-ea5e-4680-b273-5b59d9fddd88 service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:48.623800 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff966de-ea5e-4680-b273-5b59d9fddd88 service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:48.624078 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff966de-ea5e-4680-b273-5b59d9fddd88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:48.858194 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff966de-ea5e-4680-b273-5b59d9fddd88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:48.860366 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff966de-ea5e-4680-b273-5b59d9fddd88 service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:48.873795 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 713/1420] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:44:48 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:44:49.162244 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eea4f5-71ca-4ca1-a4de-f7d2e19394d4 service cinder] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:49.163669 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-38eea4f5-71ca-4ca1-a4de-f7d2e19394d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:49.224379 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ccb460-0ae8-40d0-9cd8-270f6681f518 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:49.224931 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ccb460-0ae8-40d0-9cd8-270f6681f518 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:49.224931 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ccb460-0ae8-40d0-9cd8-270f6681f518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:49.269011 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-f1ccb460-0ae8-40d0-9cd8-270f6681f518 None None] MFA Rules not processed for user `388ca829d87348bd9b524fd9a80e578b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:49.270301 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-f1ccb460-0ae8-40d0-9cd8-270f6681f518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:49.302623 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eea4f5-71ca-4ca1-a4de-f7d2e19394d4 service cinder] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:49.308810 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2bd66f-1bca-42ea-8245-4d04473dc2ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:49.309480 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2bd66f-1bca-42ea-8245-4d04473dc2ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:49.309945 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2bd66f-1bca-42ea-8245-4d04473dc2ef admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:49.310378 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2bd66f-1bca-42ea-8245-4d04473dc2ef admin admin] PATH_INFO: `/v3/users/2fe5480da9a34418a49c657c90542988` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:49.311759 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2bd66f-1bca-42ea-8245-4d04473dc2ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fe5480da9a34418a49c657c90542988)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:49.313969 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2bd66f-1bca-42ea-8245-4d04473dc2ef admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:49.409378 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:44:49.429830 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 714/1421] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:44:49 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 08:44:49.438350 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:44:49.455825 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-bb2bd66f-1bca-42ea-8245-4d04473dc2ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fe5480da9a34418a49c657c90542988'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:49.466963 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-bb2bd66f-1bca-42ea-8245-4d04473dc2ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fe5480da9a34418a49c657c90542988'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:49.522438 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-bb2bd66f-1bca-42ea-8245-4d04473dc2ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fe5480da9a34418a49c657c90542988'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:49.558210 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-bb2bd66f-1bca-42ea-8245-4d04473dc2ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fe5480da9a34418a49c657c90542988'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:49.559258 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-bb2bd66f-1bca-42ea-8245-4d04473dc2ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fe5480da9a34418a49c657c90542988'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:49.599259 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 708/1422] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 08:44:49 2022] DELETE /identity/v3/users/2fe5480da9a34418a49c657c90542988 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:44:49.604225 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef2a7c-5916-4770-8ac4-a247f3ab66af None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:49.604837 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef2a7c-5916-4770-8ac4-a247f3ab66af None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:49.605126 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef2a7c-5916-4770-8ac4-a247f3ab66af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:49.644495 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-ebef2a7c-5916-4770-8ac4-a247f3ab66af None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:49.647853 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-ebef2a7c-5916-4770-8ac4-a247f3ab66af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:49.744901 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 715/1423] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:44:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:49.750872 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2bd66f-1bca-42ea-8245-4d04473dc2ef admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:49.752342 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-bb2bd66f-1bca-42ea-8245-4d04473dc2ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:49.936462 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2bd66f-1bca-42ea-8245-4d04473dc2ef admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:49.946872 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013e312c-6275-4c28-bb14-b5277bc71f51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:49.948278 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e312c-6275-4c28-bb14-b5277bc71f51 service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:49.948569 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e312c-6275-4c28-bb14-b5277bc71f51 service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:49.949273 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e312c-6275-4c28-bb14-b5277bc71f51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:49.985500 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013e312c-6275-4c28-bb14-b5277bc71f51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:49.990270 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013e312c-6275-4c28-bb14-b5277bc71f51 service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:50.007284 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd44687-0272-4849-96e2-d8f4925e8b58 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:50.007578 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd44687-0272-4849-96e2-d8f4925e8b58 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:50.007857 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd44687-0272-4849-96e2-d8f4925e8b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:50.068957 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 709/1424] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:44:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:44:50.243162 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-6cd44687-0272-4849-96e2-d8f4925e8b58 None None] MFA Rules not processed for user `c5b2a0e623c348ad94d4820080677be6`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:50.250992 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-6cd44687-0272-4849-96e2-d8f4925e8b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:50.584847 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 716/1425] 10.222.0.30 () {58 vars in 1009 bytes} [Sat Oct 15 08:44:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 578 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:50.800440 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013e312c-6275-4c28-bb14-b5277bc71f51 service neutron] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:50.802795 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-013e312c-6275-4c28-bb14-b5277bc71f51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:51.016876 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013e312c-6275-4c28-bb14-b5277bc71f51 service neutron] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:51.046491 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f4ffb4-cd7e-4828-a3fb-f67ccf699741 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e3009e20e4c49e5a74f7fd793b2c329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:51.047291 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f4ffb4-cd7e-4828-a3fb-f67ccf699741 service swift] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:51.047591 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f4ffb4-cd7e-4828-a3fb-f67ccf699741 service swift] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:51.047898 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f4ffb4-cd7e-4828-a3fb-f67ccf699741 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:51.071036 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f4ffb4-cd7e-4828-a3fb-f67ccf699741 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:51.074775 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f4ffb4-cd7e-4828-a3fb-f67ccf699741 service swift] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:51.105834 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 710/1426] 10.222.0.30 () {60 vars in 1384 bytes} [Sat Oct 15 08:44:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 08:44:52.460766 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd44687-0272-4849-96e2-d8f4925e8b58 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:52.464932 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-6cd44687-0272-4849-96e2-d8f4925e8b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:52.540519 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd44687-0272-4849-96e2-d8f4925e8b58 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:52.572663 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a672803-8a9e-4897-a936-c0cbef07f7fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:52.573161 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a672803-8a9e-4897-a936-c0cbef07f7fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:52.573448 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a672803-8a9e-4897-a936-c0cbef07f7fc admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:52.573749 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a672803-8a9e-4897-a936-c0cbef07f7fc admin admin] PATH_INFO: `/v3/projects/d18c41fc79fe4fd2be58ca56ba77a660` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:52.577408 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a672803-8a9e-4897-a936-c0cbef07f7fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=d18c41fc79fe4fd2be58ca56ba77a660)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:52.579453 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a672803-8a9e-4897-a936-c0cbef07f7fc admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:52.797644 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-3a672803-8a9e-4897-a936-c0cbef07f7fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd18c41fc79fe4fd2be58ca56ba77a660', 'request_id': 'req-3a672803-8a9e-4897-a936-c0cbef07f7fc'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:52.833590 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-3a672803-8a9e-4897-a936-c0cbef07f7fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd18c41fc79fe4fd2be58ca56ba77a660', 'request_id': 'req-3a672803-8a9e-4897-a936-c0cbef07f7fc'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:52.875377 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-3a672803-8a9e-4897-a936-c0cbef07f7fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd18c41fc79fe4fd2be58ca56ba77a660', 'request_id': 'req-3a672803-8a9e-4897-a936-c0cbef07f7fc'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:52.886050 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:44:52.967193 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-3a672803-8a9e-4897-a936-c0cbef07f7fc admin admin] The token cache is being invalidate because project d18c41fc79fe4fd2be58ca56ba77a660 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 08:44:52.967497 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-3a672803-8a9e-4897-a936-c0cbef07f7fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:52.979602 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-3a672803-8a9e-4897-a936-c0cbef07f7fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:44:52.993102 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 717/1427] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 08:44:52 2022] DELETE /identity/v3/projects/d18c41fc79fe4fd2be58ca56ba77a660 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:44:52.999076 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b753008-3f4a-4018-a51e-da9c39ddea90 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:53.002540 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b753008-3f4a-4018-a51e-da9c39ddea90 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:53.003556 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b753008-3f4a-4018-a51e-da9c39ddea90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:53.141552 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-9b753008-3f4a-4018-a51e-da9c39ddea90 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:53.146804 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-9b753008-3f4a-4018-a51e-da9c39ddea90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:53.297650 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:44:53.452411 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 711/1428] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:44:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 449 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:53.455599 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ffb775-cc56-45dd-8bf1-2e8997b8d065 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:53.456082 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ffb775-cc56-45dd-8bf1-2e8997b8d065 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:53.456656 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ffb775-cc56-45dd-8bf1-2e8997b8d065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:53.491616 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-d0ffb775-cc56-45dd-8bf1-2e8997b8d065 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:53.497021 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-d0ffb775-cc56-45dd-8bf1-2e8997b8d065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:53.514602 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 718/1429] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:44:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:53.518218 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b753008-3f4a-4018-a51e-da9c39ddea90 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:53.537251 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b753008-3f4a-4018-a51e-da9c39ddea90 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:53.551733 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e89ec2c-778f-4e44-b93f-1c6813c52475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:53.552616 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89ec2c-778f-4e44-b93f-1c6813c52475 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:53.552992 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89ec2c-778f-4e44-b93f-1c6813c52475 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:53.553487 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89ec2c-778f-4e44-b93f-1c6813c52475 admin admin] PATH_INFO: `/v3/domains` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:53.554388 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e89ec2c-778f-4e44-b93f-1c6813c52475 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:44:53.555488 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e89ec2c-778f-4e44-b93f-1c6813c52475 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:53.564752 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e89ec2c-778f-4e44-b93f-1c6813c52475 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:53.593160 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 712/1430] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 08:44:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 08:44:53.599504 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ffb775-cc56-45dd-8bf1-2e8997b8d065 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:53.619075 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ffb775-cc56-45dd-8bf1-2e8997b8d065 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:53.627440 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79587d9a-e979-4920-a437-fd09f698775a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:53.629469 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79587d9a-e979-4920-a437-fd09f698775a admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:53.630087 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79587d9a-e979-4920-a437-fd09f698775a admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:53.630769 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79587d9a-e979-4920-a437-fd09f698775a admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:53.631857 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79587d9a-e979-4920-a437-fd09f698775a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:53.634311 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79587d9a-e979-4920-a437-fd09f698775a admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:53.733503 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 719/1431] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 08:44:53 2022] POST /identity/v3/projects => generated 377 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:44:53.739639 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e89ec2c-778f-4e44-b93f-1c6813c52475 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:53.766071 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e89ec2c-778f-4e44-b93f-1c6813c52475 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:53.918696 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d5da9d-ff38-40cc-b461-f30bd29e57d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:53.919975 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d5da9d-ff38-40cc-b461-f30bd29e57d9 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:53.922277 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d5da9d-ff38-40cc-b461-f30bd29e57d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:53.922845 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d5da9d-ff38-40cc-b461-f30bd29e57d9 admin admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:53.923736 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d5da9d-ff38-40cc-b461-f30bd29e57d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:53.926008 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d5da9d-ff38-40cc-b461-f30bd29e57d9 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:53.971677 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79587d9a-e979-4920-a437-fd09f698775a admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:53.981605 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 713/1432] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:44:53 2022] POST /identity/v3/users => generated 289 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:44:53.987858 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79587d9a-e979-4920-a437-fd09f698775a admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:53.998994 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5b5750-e041-4f51-8888-56220240a89c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:54.000547 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5b5750-e041-4f51-8888-56220240a89c admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:54.001080 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5b5750-e041-4f51-8888-56220240a89c admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:54.001633 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5b5750-e041-4f51-8888-56220240a89c admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:54.002727 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5b5750-e041-4f51-8888-56220240a89c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:54.005765 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5b5750-e041-4f51-8888-56220240a89c admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:54.062106 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d5da9d-ff38-40cc-b461-f30bd29e57d9 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:54.081732 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 720/1433] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:44:53 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:44:54.107277 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d5da9d-ff38-40cc-b461-f30bd29e57d9 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:54.130380 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e21b7-0e3d-40cc-937e-e6492cdf0d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:54.132159 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e21b7-0e3d-40cc-937e-e6492cdf0d18 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:54.134271 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e21b7-0e3d-40cc-937e-e6492cdf0d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:54.134891 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e21b7-0e3d-40cc-937e-e6492cdf0d18 admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:54.138692 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44e21b7-0e3d-40cc-937e-e6492cdf0d18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:54.144705 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44e21b7-0e3d-40cc-937e-e6492cdf0d18 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:54.191174 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 714/1434] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:44:54 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:44:54.198886 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5b5750-e041-4f51-8888-56220240a89c admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:54.224149 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5b5750-e041-4f51-8888-56220240a89c admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:54.246398 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad958c6e-2996-49ad-9e05-f0f96a57f172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:54.250612 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad958c6e-2996-49ad-9e05-f0f96a57f172 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:54.252450 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad958c6e-2996-49ad-9e05-f0f96a57f172 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:54.252703 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad958c6e-2996-49ad-9e05-f0f96a57f172 admin admin] PATH_INFO: `/v3/projects/d30615a9e44047daa7af8e9bc897681d/users/f5dd1b8ef9524ef6a5a7f19dd7b1b2be/roles/3248f0268738489b883041851d681bb7` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:54.338440 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad958c6e-2996-49ad-9e05-f0f96a57f172 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d30615a9e44047daa7af8e9bc897681d, user_id=f5dd1b8ef9524ef6a5a7f19dd7b1b2be, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:54.359185 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad958c6e-2996-49ad-9e05-f0f96a57f172 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:54.421861 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 721/1435] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:44:54 2022] PUT /identity/v3/projects/d30615a9e44047daa7af8e9bc897681d/users/f5dd1b8ef9524ef6a5a7f19dd7b1b2be/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:44:54.438923 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743847d7-40f0-4072-847e-9457c93a4b3c None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:54.439208 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743847d7-40f0-4072-847e-9457c93a4b3c None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:54.439486 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743847d7-40f0-4072-847e-9457c93a4b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:54.532662 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ee9f38-e3ba-4601-9124-7310c8aa0f9b None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:54.534603 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ee9f38-e3ba-4601-9124-7310c8aa0f9b None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:54.535043 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ee9f38-e3ba-4601-9124-7310c8aa0f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:54.539943 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-743847d7-40f0-4072-847e-9457c93a4b3c None None] MFA Rules not processed for user `f5dd1b8ef9524ef6a5a7f19dd7b1b2be`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:54.541404 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-743847d7-40f0-4072-847e-9457c93a4b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:54.612466 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-88ee9f38-e3ba-4601-9124-7310c8aa0f9b None None] MFA Rules not processed for user `8b11e349e4cb4558a4c75d843effc0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:54.615548 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-88ee9f38-e3ba-4601-9124-7310c8aa0f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:54.828423 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 715/1436] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:44:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 387 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:54.830035 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743847d7-40f0-4072-847e-9457c93a4b3c None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:54.890849 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 722/1437] 10.222.0.33 () {58 vars in 1015 bytes} [Sat Oct 15 08:44:54 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 08:44:54.906943 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:44:54.992877 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743847d7-40f0-4072-847e-9457c93a4b3c None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:55.010419 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeea640-ebf1-4189-a78c-c1480c075d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:55.011154 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeea640-ebf1-4189-a78c-c1480c075d04 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:55.011429 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeea640-ebf1-4189-a78c-c1480c075d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:55.011706 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeea640-ebf1-4189-a78c-c1480c075d04 admin admin] PATH_INFO: `/v3/projects` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:55.012580 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeea640-ebf1-4189-a78c-c1480c075d04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:55.014605 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeea640-ebf1-4189-a78c-c1480c075d04 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:55.102598 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 716/1438] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 08:44:54 2022] POST /identity/v3/projects => generated 377 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 08:44:55.108772 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ee9f38-e3ba-4601-9124-7310c8aa0f9b None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:55.171050 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ee9f38-e3ba-4601-9124-7310c8aa0f9b None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:55.370672 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a257893-1684-4def-b87b-40a3c4f58839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:55.371344 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a257893-1684-4def-b87b-40a3c4f58839 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:55.371620 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a257893-1684-4def-b87b-40a3c4f58839 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:55.371901 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a257893-1684-4def-b87b-40a3c4f58839 admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:55.372570 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a257893-1684-4def-b87b-40a3c4f58839 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:55.400009 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a257893-1684-4def-b87b-40a3c4f58839 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:55.446383 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeea640-ebf1-4189-a78c-c1480c075d04 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:55.448194 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-baeea640-ebf1-4189-a78c-c1480c075d04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:55.460407 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 723/1439] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:44:55 2022] POST /identity/v3/users => generated 295 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:44:55.464836 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a257893-1684-4def-b87b-40a3c4f58839 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:55.484383 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a257893-1684-4def-b87b-40a3c4f58839 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:55.499529 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610efe03-46ce-43f0-b2db-ed26dbc88dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:55.500203 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610efe03-46ce-43f0-b2db-ed26dbc88dac admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:55.500496 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610efe03-46ce-43f0-b2db-ed26dbc88dac admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:55.500785 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610efe03-46ce-43f0-b2db-ed26dbc88dac admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:55.501437 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610efe03-46ce-43f0-b2db-ed26dbc88dac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:55.503788 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610efe03-46ce-43f0-b2db-ed26dbc88dac admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:55.531244 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 724/1440] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:44:55 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 08:44:55.533269 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610efe03-46ce-43f0-b2db-ed26dbc88dac admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:55.562355 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610efe03-46ce-43f0-b2db-ed26dbc88dac admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:55.579171 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ccd287-6770-40ef-b92d-c7335604d38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:55.580677 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ccd287-6770-40ef-b92d-c7335604d38b admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:55.581163 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ccd287-6770-40ef-b92d-c7335604d38b admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:55.581604 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ccd287-6770-40ef-b92d-c7335604d38b admin admin] PATH_INFO: `/v3/projects/8a6a6477e96640d29aa86f602ffd3273/users/f5d5dbc016b94551be445e7f5e40d480/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:55.644380 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=94,keystone:DELETE=1,keystone:INSERT=5 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:44:55.654014 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ccd287-6770-40ef-b92d-c7335604d38b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a6a6477e96640d29aa86f602ffd3273, user_id=f5d5dbc016b94551be445e7f5e40d480, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:55.657406 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ccd287-6770-40ef-b92d-c7335604d38b admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:55.659926 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeea640-ebf1-4189-a78c-c1480c075d04 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:55.665229 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af400d14-bd7c-48d9-a25d-7d9fb3352176 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:55.665909 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af400d14-bd7c-48d9-a25d-7d9fb3352176 service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:55.666200 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af400d14-bd7c-48d9-a25d-7d9fb3352176 service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:55.666483 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af400d14-bd7c-48d9-a25d-7d9fb3352176 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:55.690399 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af400d14-bd7c-48d9-a25d-7d9fb3352176 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:55.692316 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af400d14-bd7c-48d9-a25d-7d9fb3352176 service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:55.699551 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 725/1441] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:44:55 2022] PUT /identity/v3/projects/8a6a6477e96640d29aa86f602ffd3273/users/f5d5dbc016b94551be445e7f5e40d480/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:44:55.706690 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ccd287-6770-40ef-b92d-c7335604d38b admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:55.708786 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 717/1442] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:44:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:44:55.949667 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ccd287-6770-40ef-b92d-c7335604d38b admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:55.967927 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1f3443-11f8-4224-a595-50e3ee695843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:55.968589 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1f3443-11f8-4224-a595-50e3ee695843 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:55.968874 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1f3443-11f8-4224-a595-50e3ee695843 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:55.969222 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1f3443-11f8-4224-a595-50e3ee695843 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:55.969919 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1f3443-11f8-4224-a595-50e3ee695843 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:55.984129 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1f3443-11f8-4224-a595-50e3ee695843 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:56.028024 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 726/1443] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:44:55 2022] GET /identity/v3/roles => generated 1420 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:44:56.035611 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af400d14-bd7c-48d9-a25d-7d9fb3352176 service neutron] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:56.095630 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af400d14-bd7c-48d9-a25d-7d9fb3352176 service neutron] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:56.117140 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c928a136-9e7d-477d-9d4b-d1ee2df024d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:56.119811 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c928a136-9e7d-477d-9d4b-d1ee2df024d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:56.123899 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c928a136-9e7d-477d-9d4b-d1ee2df024d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:56.124078 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c928a136-9e7d-477d-9d4b-d1ee2df024d9 admin admin] PATH_INFO: `/v3/projects/8a6a6477e96640d29aa86f602ffd3273/users/f5d5dbc016b94551be445e7f5e40d480/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:56.130600 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c928a136-9e7d-477d-9d4b-d1ee2df024d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a6a6477e96640d29aa86f602ffd3273, user_id=f5d5dbc016b94551be445e7f5e40d480, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:56.136973 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c928a136-9e7d-477d-9d4b-d1ee2df024d9 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:56.183544 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 718/1444] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:44:56 2022] PUT /identity/v3/projects/8a6a6477e96640d29aa86f602ffd3273/users/f5d5dbc016b94551be445e7f5e40d480/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:44:56.194997 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b08672f-c7de-401f-a816-4c582acc77be None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:56.195296 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b08672f-c7de-401f-a816-4c582acc77be None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:56.195821 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b08672f-c7de-401f-a816-4c582acc77be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:56.278468 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-0b08672f-c7de-401f-a816-4c582acc77be None None] MFA Rules not processed for user `f5d5dbc016b94551be445e7f5e40d480`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:44:56.297574 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-0b08672f-c7de-401f-a816-4c582acc77be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:56.301946 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c928a136-9e7d-477d-9d4b-d1ee2df024d9 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:56.303647 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-c928a136-9e7d-477d-9d4b-d1ee2df024d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:56.529002 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c928a136-9e7d-477d-9d4b-d1ee2df024d9 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:56.547618 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2941210e-af38-4860-a973-019adf79e3ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:56.548550 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2941210e-af38-4860-a973-019adf79e3ed service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:56.549015 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2941210e-af38-4860-a973-019adf79e3ed service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:56.549213 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2941210e-af38-4860-a973-019adf79e3ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:56.575780 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2941210e-af38-4860-a973-019adf79e3ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:56.578854 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2941210e-af38-4860-a973-019adf79e3ed service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:56.600763 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 727/1445] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:44:56 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 407 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:44:56.615171 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b08672f-c7de-401f-a816-4c582acc77be None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:44:56.618257 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 719/1446] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:44:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 08:44:56.620722 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-0b08672f-c7de-401f-a816-4c582acc77be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:44:56.817021 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b08672f-c7de-401f-a816-4c582acc77be None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:44:56.841422 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c587c8-a751-4b4a-af01-6ca2b3eb6efe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:44:56.842165 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c587c8-a751-4b4a-af01-6ca2b3eb6efe service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:44:56.844078 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c587c8-a751-4b4a-af01-6ca2b3eb6efe service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:44:56.844375 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c587c8-a751-4b4a-af01-6ca2b3eb6efe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:44:56.951377 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:44:57.065561 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c587c8-a751-4b4a-af01-6ca2b3eb6efe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:44:57.068592 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c587c8-a751-4b4a-af01-6ca2b3eb6efe service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:44:57.088046 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 728/1447] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:44:56 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:45:00.316211 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b6fb35-c2c1-48a3-b715-604fdf9016fa None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:00.316817 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b6fb35-c2c1-48a3-b715-604fdf9016fa None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:00.316904 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b6fb35-c2c1-48a3-b715-604fdf9016fa None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:00.318562 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 720/1448] 10.222.0.30 () {54 vars in 921 bytes} [Sat Oct 15 08:45:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:45:00.699996 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd11cfa4-7ac5-42f7-9e15-b399d8d88758 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:00.700566 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd11cfa4-7ac5-42f7-9e15-b399d8d88758 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:00.700566 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd11cfa4-7ac5-42f7-9e15-b399d8d88758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:00.761518 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-bd11cfa4-7ac5-42f7-9e15-b399d8d88758 None None] MFA Rules not processed for user `af0a8af5cadc4cc1ab73e8eb2a20bb7e`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:00.766387 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-bd11cfa4-7ac5-42f7-9e15-b399d8d88758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:00.886118 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0fa1b1-a56b-4398-91a7-ea933241c2e3 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:00.886234 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0fa1b1-a56b-4398-91a7-ea933241c2e3 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:00.886476 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0fa1b1-a56b-4398-91a7-ea933241c2e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:00.903307 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 729/1449] 10.222.0.30 () {58 vars in 1018 bytes} [Sat Oct 15 08:45:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:45:00.934749 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-af0fa1b1-a56b-4398-91a7-ea933241c2e3 None None] MFA Rules not processed for user `060d1fae9b0b488da1f7553cabd97d28`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:00.944707 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-af0fa1b1-a56b-4398-91a7-ea933241c2e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:01.074671 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd11cfa4-7ac5-42f7-9e15-b399d8d88758 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:01.077025 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-bd11cfa4-7ac5-42f7-9e15-b399d8d88758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:01.103178 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd11cfa4-7ac5-42f7-9e15-b399d8d88758 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:01.111348 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:01.122154 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb99fdeb-5af4-4605-bd76-a3ddcef7a0f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0a8af5cadc4cc1ab73e8eb2a20bb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:01.123434 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb99fdeb-5af4-4605-bd76-a3ddcef7a0f7 service nova] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:01.123730 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb99fdeb-5af4-4605-bd76-a3ddcef7a0f7 service nova] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:01.124508 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb99fdeb-5af4-4605-bd76-a3ddcef7a0f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:01.130756 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 721/1450] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 08:45:01.136964 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb99fdeb-5af4-4605-bd76-a3ddcef7a0f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:01.138859 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb99fdeb-5af4-4605-bd76-a3ddcef7a0f7 service nova] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:01.165038 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 730/1451] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:45:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:45:01.988323 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bd9dda-be50-4922-a9c0-fd15d2b00050 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:01.989256 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bd9dda-be50-4922-a9c0-fd15d2b00050 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:01.989748 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bd9dda-be50-4922-a9c0-fd15d2b00050 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:01.991998 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 722/1452] 10.222.0.30 () {54 vars in 921 bytes} [Sat Oct 15 08:45:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:45:02.033121 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba984f32-1b48-4bcf-9e4d-6b151799e666 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:02.033662 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba984f32-1b48-4bcf-9e4d-6b151799e666 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:02.033662 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba984f32-1b48-4bcf-9e4d-6b151799e666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:02.073991 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-ba984f32-1b48-4bcf-9e4d-6b151799e666 None None] MFA Rules not processed for user `af0a8af5cadc4cc1ab73e8eb2a20bb7e`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:02.075493 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-ba984f32-1b48-4bcf-9e4d-6b151799e666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:02.096418 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 731/1453] 10.222.0.30 () {58 vars in 1018 bytes} [Sat Oct 15 08:45:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:45:02.711695 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bd9dda-be50-4922-a9c0-fd15d2b00050 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:02.734787 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bd9dda-be50-4922-a9c0-fd15d2b00050 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:02.741130 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd7ba70-5f21-4a02-8900-575467049cc9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0a8af5cadc4cc1ab73e8eb2a20bb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:02.742313 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd7ba70-5f21-4a02-8900-575467049cc9 service nova] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:02.742756 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd7ba70-5f21-4a02-8900-575467049cc9 service nova] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:02.743178 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd7ba70-5f21-4a02-8900-575467049cc9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:02.777605 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd7ba70-5f21-4a02-8900-575467049cc9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:02.779794 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd7ba70-5f21-4a02-8900-575467049cc9 service nova] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:02.796552 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 723/1454] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:45:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:45:03.495016 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba984f32-1b48-4bcf-9e4d-6b151799e666 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:03.519415 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba984f32-1b48-4bcf-9e4d-6b151799e666 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:03.532243 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a241d4c-73d0-41dc-a2a7-48a1578956e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:03.532846 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a241d4c-73d0-41dc-a2a7-48a1578956e6 service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:03.533376 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a241d4c-73d0-41dc-a2a7-48a1578956e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:03.533665 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a241d4c-73d0-41dc-a2a7-48a1578956e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:03.554925 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a241d4c-73d0-41dc-a2a7-48a1578956e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:03.559675 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a241d4c-73d0-41dc-a2a7-48a1578956e6 service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:03.589236 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 732/1455] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:45:03 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:45:12.791106 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=9 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:13.575933 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=61 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:18.609859 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd7ba70-5f21-4a02-8900-575467049cc9 service nova] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:18.612129 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-edd7ba70-5f21-4a02-8900-575467049cc9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:18.696904 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd7ba70-5f21-4a02-8900-575467049cc9 service nova] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:18.702584 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a447e8-2e1e-4d42-92d6-c00bb89fa553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:18.703297 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a447e8-2e1e-4d42-92d6-c00bb89fa553 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:18.703586 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a447e8-2e1e-4d42-92d6-c00bb89fa553 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:18.703860 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a447e8-2e1e-4d42-92d6-c00bb89fa553 admin admin] PATH_INFO: `/v3/users/657a8bdd2b834ba9b53c0d8bf47a9e19` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:18.705014 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a447e8-2e1e-4d42-92d6-c00bb89fa553 admin admin] RBAC: Authorizing `identity:delete_user(user_id=657a8bdd2b834ba9b53c0d8bf47a9e19)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:18.707007 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a447e8-2e1e-4d42-92d6-c00bb89fa553 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:18.764967 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-b7a447e8-2e1e-4d42-92d6-c00bb89fa553 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '657a8bdd2b834ba9b53c0d8bf47a9e19'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:18.765724 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-b7a447e8-2e1e-4d42-92d6-c00bb89fa553 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '657a8bdd2b834ba9b53c0d8bf47a9e19'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:18.780759 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-b7a447e8-2e1e-4d42-92d6-c00bb89fa553 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '657a8bdd2b834ba9b53c0d8bf47a9e19'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:18.789561 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-b7a447e8-2e1e-4d42-92d6-c00bb89fa553 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '657a8bdd2b834ba9b53c0d8bf47a9e19'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:18.790256 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-b7a447e8-2e1e-4d42-92d6-c00bb89fa553 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '657a8bdd2b834ba9b53c0d8bf47a9e19'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:18.805978 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 724/1456] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 08:45:18 2022] DELETE /identity/v3/users/657a8bdd2b834ba9b53c0d8bf47a9e19 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:18.811538 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1e708-3373-4068-856f-837d08d67b67 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:18.813318 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1e708-3373-4068-856f-837d08d67b67 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:18.813992 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1e708-3373-4068-856f-837d08d67b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:18.849360 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-f8c1e708-3373-4068-856f-837d08d67b67 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:18.850997 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-f8c1e708-3373-4068-856f-837d08d67b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:18.959815 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 733/1457] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:45:18.969096 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a447e8-2e1e-4d42-92d6-c00bb89fa553 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:18.971301 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-b7a447e8-2e1e-4d42-92d6-c00bb89fa553 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:19.083968 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a447e8-2e1e-4d42-92d6-c00bb89fa553 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:19.088980 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7feaef-12d4-43bb-a7a6-ad64348b8add service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:19.089711 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7feaef-12d4-43bb-a7a6-ad64348b8add service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:19.090172 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7feaef-12d4-43bb-a7a6-ad64348b8add service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:19.090523 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7feaef-12d4-43bb-a7a6-ad64348b8add service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:19.100735 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7feaef-12d4-43bb-a7a6-ad64348b8add service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:19.102396 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7feaef-12d4-43bb-a7a6-ad64348b8add service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:19.116673 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 725/1458] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:45:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:45:19.621190 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c1e708-3373-4068-856f-837d08d67b67 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:19.622943 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-f8c1e708-3373-4068-856f-837d08d67b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:19.644472 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c1e708-3373-4068-856f-837d08d67b67 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:19.652004 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0511fc-be0e-4943-a8f3-a5ebbe9a6f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:19.652920 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0511fc-be0e-4943-a8f3-a5ebbe9a6f93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:19.653365 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0511fc-be0e-4943-a8f3-a5ebbe9a6f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:19.653932 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0511fc-be0e-4943-a8f3-a5ebbe9a6f93 admin admin] PATH_INFO: `/v3/projects/24068ef2565945b0bc34a801c9e2b3d8` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:19.656390 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0511fc-be0e-4943-a8f3-a5ebbe9a6f93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24068ef2565945b0bc34a801c9e2b3d8)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:19.659191 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0511fc-be0e-4943-a8f3-a5ebbe9a6f93 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:19.845523 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-1b0511fc-be0e-4943-a8f3-a5ebbe9a6f93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24068ef2565945b0bc34a801c9e2b3d8', 'request_id': 'req-1b0511fc-be0e-4943-a8f3-a5ebbe9a6f93'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:19.858692 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-1b0511fc-be0e-4943-a8f3-a5ebbe9a6f93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24068ef2565945b0bc34a801c9e2b3d8', 'request_id': 'req-1b0511fc-be0e-4943-a8f3-a5ebbe9a6f93'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:19.875610 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-1b0511fc-be0e-4943-a8f3-a5ebbe9a6f93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24068ef2565945b0bc34a801c9e2b3d8', 'request_id': 'req-1b0511fc-be0e-4943-a8f3-a5ebbe9a6f93'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:19.896782 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-1b0511fc-be0e-4943-a8f3-a5ebbe9a6f93 admin admin] The token cache is being invalidate because project 24068ef2565945b0bc34a801c9e2b3d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 08:45:19.897338 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-1b0511fc-be0e-4943-a8f3-a5ebbe9a6f93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:19.899753 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-1b0511fc-be0e-4943-a8f3-a5ebbe9a6f93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:19.903154 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 734/1459] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 08:45:19 2022] DELETE /identity/v3/projects/24068ef2565945b0bc34a801c9e2b3d8 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:19.906923 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7feaef-12d4-43bb-a7a6-ad64348b8add service neutron] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:19.918876 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7feaef-12d4-43bb-a7a6-ad64348b8add service neutron] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:19.948892 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:20.013213 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11737838-18d9-474e-98c5-1aa847baf9c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:20.014201 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11737838-18d9-474e-98c5-1aa847baf9c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:20.014807 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11737838-18d9-474e-98c5-1aa847baf9c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:20.015142 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11737838-18d9-474e-98c5-1aa847baf9c4 admin admin] PATH_INFO: `/v3/users/c6c9025530064170b7b18c0cc2e0dca2` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:20.016547 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11737838-18d9-474e-98c5-1aa847baf9c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6c9025530064170b7b18c0cc2e0dca2)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:20.019550 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11737838-18d9-474e-98c5-1aa847baf9c4 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:20.085417 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-11737838-18d9-474e-98c5-1aa847baf9c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6c9025530064170b7b18c0cc2e0dca2'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:20.086209 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-11737838-18d9-474e-98c5-1aa847baf9c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6c9025530064170b7b18c0cc2e0dca2'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:20.102745 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-11737838-18d9-474e-98c5-1aa847baf9c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6c9025530064170b7b18c0cc2e0dca2'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:20.113973 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-11737838-18d9-474e-98c5-1aa847baf9c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6c9025530064170b7b18c0cc2e0dca2'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:20.114866 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-11737838-18d9-474e-98c5-1aa847baf9c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6c9025530064170b7b18c0cc2e0dca2'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:20.129208 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 726/1460] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 08:45:19 2022] DELETE /identity/v3/users/c6c9025530064170b7b18c0cc2e0dca2 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:20.460018 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0511fc-be0e-4943-a8f3-a5ebbe9a6f93 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:20.461582 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-1b0511fc-be0e-4943-a8f3-a5ebbe9a6f93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:20.495222 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0511fc-be0e-4943-a8f3-a5ebbe9a6f93 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:20.577886 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94e85fc-f5a4-489d-9644-fbafd6575d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:20.578468 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94e85fc-f5a4-489d-9644-fbafd6575d72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:20.578750 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94e85fc-f5a4-489d-9644-fbafd6575d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:20.579027 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94e85fc-f5a4-489d-9644-fbafd6575d72 admin admin] PATH_INFO: `/v3/projects/43b43c8c7421405083709d60ab899424` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:20.580178 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94e85fc-f5a4-489d-9644-fbafd6575d72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43b43c8c7421405083709d60ab899424)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:20.582153 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94e85fc-f5a4-489d-9644-fbafd6575d72 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:20.615930 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:20.636445 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-a94e85fc-f5a4-489d-9644-fbafd6575d72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43b43c8c7421405083709d60ab899424', 'request_id': 'req-a94e85fc-f5a4-489d-9644-fbafd6575d72'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:20.643422 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-a94e85fc-f5a4-489d-9644-fbafd6575d72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43b43c8c7421405083709d60ab899424', 'request_id': 'req-a94e85fc-f5a4-489d-9644-fbafd6575d72'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:20.654531 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-a94e85fc-f5a4-489d-9644-fbafd6575d72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43b43c8c7421405083709d60ab899424', 'request_id': 'req-a94e85fc-f5a4-489d-9644-fbafd6575d72'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:20.666480 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-a94e85fc-f5a4-489d-9644-fbafd6575d72 admin admin] The token cache is being invalidate because project 43b43c8c7421405083709d60ab899424 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 08:45:20.666776 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-a94e85fc-f5a4-489d-9644-fbafd6575d72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:20.667441 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-a94e85fc-f5a4-489d-9644-fbafd6575d72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:20.668947 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 735/1461] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 08:45:20 2022] DELETE /identity/v3/projects/43b43c8c7421405083709d60ab899424 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:20.674087 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a39054d-374c-4e22-8fe0-0eec9697581f None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:20.674387 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a39054d-374c-4e22-8fe0-0eec9697581f None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:20.674663 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a39054d-374c-4e22-8fe0-0eec9697581f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:20.694481 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-8a39054d-374c-4e22-8fe0-0eec9697581f None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:20.695773 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-8a39054d-374c-4e22-8fe0-0eec9697581f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:20.756216 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 727/1462] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:45:20.764193 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59309058-5ba8-4284-ba47-6e3d004693e9 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:20.764475 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59309058-5ba8-4284-ba47-6e3d004693e9 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:20.764745 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59309058-5ba8-4284-ba47-6e3d004693e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:20.785328 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-59309058-5ba8-4284-ba47-6e3d004693e9 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:20.786593 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-59309058-5ba8-4284-ba47-6e3d004693e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:20.795408 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 736/1463] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:45:20.799588 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a39054d-374c-4e22-8fe0-0eec9697581f None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:20.812985 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a39054d-374c-4e22-8fe0-0eec9697581f None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:20.826492 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4aad73-6f92-486f-b593-ae4c8c280425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:20.827190 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4aad73-6f92-486f-b593-ae4c8c280425 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:20.827615 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4aad73-6f92-486f-b593-ae4c8c280425 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:20.827815 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4aad73-6f92-486f-b593-ae4c8c280425 admin admin] PATH_INFO: `/v3/domains` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:20.828508 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4aad73-6f92-486f-b593-ae4c8c280425 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:45:20.828840 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4aad73-6f92-486f-b593-ae4c8c280425 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:20.838292 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4aad73-6f92-486f-b593-ae4c8c280425 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:20.843831 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 728/1464] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 08:45:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:45:20.848119 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59309058-5ba8-4284-ba47-6e3d004693e9 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:20.860902 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59309058-5ba8-4284-ba47-6e3d004693e9 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:20.867134 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85786ef6-3d58-470d-9990-128834e68034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:20.867742 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85786ef6-3d58-470d-9990-128834e68034 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:20.868179 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85786ef6-3d58-470d-9990-128834e68034 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:20.868610 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85786ef6-3d58-470d-9990-128834e68034 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:20.869402 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85786ef6-3d58-470d-9990-128834e68034 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:20.871493 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85786ef6-3d58-470d-9990-128834e68034 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:20.904254 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 737/1465] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 08:45:20 2022] POST /identity/v3/projects => generated 379 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:45:20.908525 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4aad73-6f92-486f-b593-ae4c8c280425 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:20.918823 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4aad73-6f92-486f-b593-ae4c8c280425 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:20.949836 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:20.988890 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e2285f-ab6b-4ae5-9b84-bd4058d70b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:20.989601 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2285f-ab6b-4ae5-9b84-bd4058d70b85 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:20.990016 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2285f-ab6b-4ae5-9b84-bd4058d70b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:20.990392 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2285f-ab6b-4ae5-9b84-bd4058d70b85 admin admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:20.991301 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e2285f-ab6b-4ae5-9b84-bd4058d70b85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:20.993636 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e2285f-ab6b-4ae5-9b84-bd4058d70b85 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:21.020079 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 729/1466] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:45:20 2022] POST /identity/v3/users => generated 290 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:45:21.021206 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85786ef6-3d58-470d-9990-128834e68034 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:21.032546 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85786ef6-3d58-470d-9990-128834e68034 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:21.037522 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b862b83-1291-45a6-8d02-2b406b79861b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:21.038231 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b862b83-1291-45a6-8d02-2b406b79861b admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:21.038516 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b862b83-1291-45a6-8d02-2b406b79861b admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:21.039353 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b862b83-1291-45a6-8d02-2b406b79861b admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:21.040024 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b862b83-1291-45a6-8d02-2b406b79861b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:21.041838 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b862b83-1291-45a6-8d02-2b406b79861b admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:21.054970 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 738/1467] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:45:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 08:45:21.057429 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e2285f-ab6b-4ae5-9b84-bd4058d70b85 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:21.068241 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e2285f-ab6b-4ae5-9b84-bd4058d70b85 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:21.073140 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8069bd-4ae4-41e9-9db9-bee3633585da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:21.073786 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8069bd-4ae4-41e9-9db9-bee3633585da admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:21.074073 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8069bd-4ae4-41e9-9db9-bee3633585da admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:21.074383 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8069bd-4ae4-41e9-9db9-bee3633585da admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:21.075063 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8069bd-4ae4-41e9-9db9-bee3633585da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:21.076907 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8069bd-4ae4-41e9-9db9-bee3633585da admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:21.088984 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 730/1468] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:45:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:45:21.094130 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b862b83-1291-45a6-8d02-2b406b79861b admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:21.105303 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b862b83-1291-45a6-8d02-2b406b79861b admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:21.111693 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4abb44-0135-4e32-9e05-6665c7558b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:21.112494 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4abb44-0135-4e32-9e05-6665c7558b3a admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:21.112779 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4abb44-0135-4e32-9e05-6665c7558b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:21.113574 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4abb44-0135-4e32-9e05-6665c7558b3a admin admin] PATH_INFO: `/v3/projects/ac4e88c95eab423c90afec990252a701/users/57a48343bb0e470b8657e81c4e990f1c/roles/3248f0268738489b883041851d681bb7` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:21.135423 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4abb44-0135-4e32-9e05-6665c7558b3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac4e88c95eab423c90afec990252a701, user_id=57a48343bb0e470b8657e81c4e990f1c, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:21.138330 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4abb44-0135-4e32-9e05-6665c7558b3a admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:21.152718 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 739/1469] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:45:21 2022] PUT /identity/v3/projects/ac4e88c95eab423c90afec990252a701/users/57a48343bb0e470b8657e81c4e990f1c/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:21.164784 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a8777-2059-487d-8907-e2b95bc18ce8 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:21.165292 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a8777-2059-487d-8907-e2b95bc18ce8 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:21.165724 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a8777-2059-487d-8907-e2b95bc18ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:21.197852 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-0a4a8777-2059-487d-8907-e2b95bc18ce8 None None] MFA Rules not processed for user `57a48343bb0e470b8657e81c4e990f1c`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:21.199143 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-0a4a8777-2059-487d-8907-e2b95bc18ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:21.276263 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 731/1470] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:45:21.280885 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4abb44-0135-4e32-9e05-6665c7558b3a admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:21.342454 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4abb44-0135-4e32-9e05-6665c7558b3a admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:21.347020 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85fd682-827b-43a1-894c-db0c414df6c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:21.347623 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85fd682-827b-43a1-894c-db0c414df6c8 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:21.347907 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85fd682-827b-43a1-894c-db0c414df6c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:21.348182 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85fd682-827b-43a1-894c-db0c414df6c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:21.348821 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85fd682-827b-43a1-894c-db0c414df6c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:21.350783 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85fd682-827b-43a1-894c-db0c414df6c8 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:21.381407 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 740/1471] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 08:45:21 2022] POST /identity/v3/projects => generated 381 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:45:21.386011 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4a8777-2059-487d-8907-e2b95bc18ce8 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:21.410876 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4a8777-2059-487d-8907-e2b95bc18ce8 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:21.476445 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045f7d46-e682-4434-9324-e7373bc95c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:21.477068 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f7d46-e682-4434-9324-e7373bc95c78 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:21.477355 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f7d46-e682-4434-9324-e7373bc95c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:21.477636 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f7d46-e682-4434-9324-e7373bc95c78 admin admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:21.478348 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045f7d46-e682-4434-9324-e7373bc95c78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:21.480404 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045f7d46-e682-4434-9324-e7373bc95c78 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:21.503892 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 732/1472] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:45:21 2022] POST /identity/v3/users => generated 297 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:45:21.508597 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85fd682-827b-43a1-894c-db0c414df6c8 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:21.519988 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85fd682-827b-43a1-894c-db0c414df6c8 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:21.525610 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b3c841-b43b-4bbc-ac72-48b852bfcda2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:21.526329 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b3c841-b43b-4bbc-ac72-48b852bfcda2 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:21.526677 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b3c841-b43b-4bbc-ac72-48b852bfcda2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:21.527071 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b3c841-b43b-4bbc-ac72-48b852bfcda2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:21.527778 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b3c841-b43b-4bbc-ac72-48b852bfcda2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:21.529490 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b3c841-b43b-4bbc-ac72-48b852bfcda2 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:21.540207 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 741/1473] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:45:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 08:45:21.544717 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045f7d46-e682-4434-9324-e7373bc95c78 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:21.555487 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045f7d46-e682-4434-9324-e7373bc95c78 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:21.560875 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471775fd-a132-42b1-a980-f1ba6ccc23ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:21.561541 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471775fd-a132-42b1-a980-f1ba6ccc23ff admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:21.561897 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471775fd-a132-42b1-a980-f1ba6ccc23ff admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:21.562201 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471775fd-a132-42b1-a980-f1ba6ccc23ff admin admin] PATH_INFO: `/v3/projects/6c44dfc9d2a9488894c53429f2250026/users/cbf2d6e9a9b14ccb852f0881d7ad8e31/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:21.577958 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471775fd-a132-42b1-a980-f1ba6ccc23ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c44dfc9d2a9488894c53429f2250026, user_id=cbf2d6e9a9b14ccb852f0881d7ad8e31, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:21.580637 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471775fd-a132-42b1-a980-f1ba6ccc23ff admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:21.598299 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 733/1474] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:45:21 2022] PUT /identity/v3/projects/6c44dfc9d2a9488894c53429f2250026/users/cbf2d6e9a9b14ccb852f0881d7ad8e31/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:21.603049 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b3c841-b43b-4bbc-ac72-48b852bfcda2 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:21.613105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b3c841-b43b-4bbc-ac72-48b852bfcda2 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:21.633774 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:21.672415 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313ebe97-520b-4764-9c44-dc9f19f87eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:21.673033 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ebe97-520b-4764-9c44-dc9f19f87eef admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:21.673347 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ebe97-520b-4764-9c44-dc9f19f87eef admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:21.673628 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ebe97-520b-4764-9c44-dc9f19f87eef admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:21.674300 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313ebe97-520b-4764-9c44-dc9f19f87eef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:21.675917 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313ebe97-520b-4764-9c44-dc9f19f87eef admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:21.686333 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 742/1475] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:45:21 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:45:21.690520 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471775fd-a132-42b1-a980-f1ba6ccc23ff admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:21.695964 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:21.702490 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471775fd-a132-42b1-a980-f1ba6ccc23ff admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:21.707522 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f6bd51-362f-4c2b-94ce-bfade933ad8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:21.708186 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6bd51-362f-4c2b-94ce-bfade933ad8e admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:21.708469 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6bd51-362f-4c2b-94ce-bfade933ad8e admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:21.708745 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6bd51-362f-4c2b-94ce-bfade933ad8e admin admin] PATH_INFO: `/v3/projects/6c44dfc9d2a9488894c53429f2250026/users/cbf2d6e9a9b14ccb852f0881d7ad8e31/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:21.710751 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f6bd51-362f-4c2b-94ce-bfade933ad8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c44dfc9d2a9488894c53429f2250026, user_id=cbf2d6e9a9b14ccb852f0881d7ad8e31, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:21.713386 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f6bd51-362f-4c2b-94ce-bfade933ad8e admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:21.724227 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 734/1476] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:45:21 2022] PUT /identity/v3/projects/6c44dfc9d2a9488894c53429f2250026/users/cbf2d6e9a9b14ccb852f0881d7ad8e31/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:21.735063 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351244b3-75a8-4bd7-b5a6-c0f82908a43d None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:21.735359 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351244b3-75a8-4bd7-b5a6-c0f82908a43d None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:21.735643 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351244b3-75a8-4bd7-b5a6-c0f82908a43d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:21.764709 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-351244b3-75a8-4bd7-b5a6-c0f82908a43d None None] MFA Rules not processed for user `cbf2d6e9a9b14ccb852f0881d7ad8e31`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:21.765964 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-351244b3-75a8-4bd7-b5a6-c0f82908a43d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:21.833414 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 743/1477] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:21 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:45:21.845981 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f6bd51-362f-4c2b-94ce-bfade933ad8e admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:21.847725 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-98f6bd51-362f-4c2b-94ce-bfade933ad8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:21.904429 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f6bd51-362f-4c2b-94ce-bfade933ad8e admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:21.908033 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230b7b9c-8dc0-44f8-9670-32e8d6fb06ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:21.908633 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230b7b9c-8dc0-44f8-9670-32e8d6fb06ff service cinder] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:21.909160 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230b7b9c-8dc0-44f8-9670-32e8d6fb06ff service cinder] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:21.909160 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230b7b9c-8dc0-44f8-9670-32e8d6fb06ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:21.918447 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230b7b9c-8dc0-44f8-9670-32e8d6fb06ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:21.919857 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230b7b9c-8dc0-44f8-9670-32e8d6fb06ff service cinder] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:21.929033 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 735/1478] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:45:21 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:45:22.256151 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351244b3-75a8-4bd7-b5a6-c0f82908a43d None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:22.265544 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351244b3-75a8-4bd7-b5a6-c0f82908a43d None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:22.270137 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b564a05-72fe-4081-a48a-1a7f76306092 tempest-VolumeQuotaClassesTest-2013247953 tempest-VolumeQuotaClassesTest-2013247953-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbf2d6e9a9b14ccb852f0881d7ad8e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c44dfc9d2a9488894c53429f2250026', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:22.270740 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b564a05-72fe-4081-a48a-1a7f76306092 tempest-VolumeQuotaClassesTest-2013247953 tempest-VolumeQuotaClassesTest-2013247953-project-admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:22.271027 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b564a05-72fe-4081-a48a-1a7f76306092 tempest-VolumeQuotaClassesTest-2013247953 tempest-VolumeQuotaClassesTest-2013247953-project-admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:22.271308 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b564a05-72fe-4081-a48a-1a7f76306092 tempest-VolumeQuotaClassesTest-2013247953 tempest-VolumeQuotaClassesTest-2013247953-project-admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:22.271951 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b564a05-72fe-4081-a48a-1a7f76306092 tempest-VolumeQuotaClassesTest-2013247953 tempest-VolumeQuotaClassesTest-2013247953-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:45:22.272274 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b564a05-72fe-4081-a48a-1a7f76306092 tempest-VolumeQuotaClassesTest-2013247953 tempest-VolumeQuotaClassesTest-2013247953-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:22.274890 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b564a05-72fe-4081-a48a-1a7f76306092 tempest-VolumeQuotaClassesTest-2013247953 tempest-VolumeQuotaClassesTest-2013247953-project-admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:22.286779 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 744/1479] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 08:45:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:45:22.291799 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230b7b9c-8dc0-44f8-9670-32e8d6fb06ff service cinder] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:22.301018 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230b7b9c-8dc0-44f8-9670-32e8d6fb06ff service cinder] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:22.305738 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26551dcc-f813-4413-a4f3-ab7641e7a8d6 tempest-VolumeQuotaClassesTest-2013247953 tempest-VolumeQuotaClassesTest-2013247953-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbf2d6e9a9b14ccb852f0881d7ad8e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c44dfc9d2a9488894c53429f2250026', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:22.306326 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26551dcc-f813-4413-a4f3-ab7641e7a8d6 tempest-VolumeQuotaClassesTest-2013247953 tempest-VolumeQuotaClassesTest-2013247953-project-admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:22.306617 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26551dcc-f813-4413-a4f3-ab7641e7a8d6 tempest-VolumeQuotaClassesTest-2013247953 tempest-VolumeQuotaClassesTest-2013247953-project-admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:22.306901 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26551dcc-f813-4413-a4f3-ab7641e7a8d6 tempest-VolumeQuotaClassesTest-2013247953 tempest-VolumeQuotaClassesTest-2013247953-project-admin] PATH_INFO: `/v3/projects` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:22.307541 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26551dcc-f813-4413-a4f3-ab7641e7a8d6 tempest-VolumeQuotaClassesTest-2013247953 tempest-VolumeQuotaClassesTest-2013247953-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:22.309449 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26551dcc-f813-4413-a4f3-ab7641e7a8d6 tempest-VolumeQuotaClassesTest-2013247953 tempest-VolumeQuotaClassesTest-2013247953-project-admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:22.344163 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 736/1480] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 08:45:22 2022] POST /identity/v3/projects => generated 353 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:45:22.348436 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b564a05-72fe-4081-a48a-1a7f76306092 tempest-VolumeQuotaClassesTest-2013247953 tempest-VolumeQuotaClassesTest-2013247953-project-admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:22.358323 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b564a05-72fe-4081-a48a-1a7f76306092 tempest-VolumeQuotaClassesTest-2013247953 tempest-VolumeQuotaClassesTest-2013247953-project-admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:22.418551 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db54c63-b876-494f-87d2-e09fa020ad59 tempest-VolumeQuotaClassesTest-2013247953 tempest-VolumeQuotaClassesTest-2013247953-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbf2d6e9a9b14ccb852f0881d7ad8e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c44dfc9d2a9488894c53429f2250026', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:22.419165 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db54c63-b876-494f-87d2-e09fa020ad59 tempest-VolumeQuotaClassesTest-2013247953 tempest-VolumeQuotaClassesTest-2013247953-project-admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:22.419450 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db54c63-b876-494f-87d2-e09fa020ad59 tempest-VolumeQuotaClassesTest-2013247953 tempest-VolumeQuotaClassesTest-2013247953-project-admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:22.419728 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db54c63-b876-494f-87d2-e09fa020ad59 tempest-VolumeQuotaClassesTest-2013247953 tempest-VolumeQuotaClassesTest-2013247953-project-admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:22.420377 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db54c63-b876-494f-87d2-e09fa020ad59 tempest-VolumeQuotaClassesTest-2013247953 tempest-VolumeQuotaClassesTest-2013247953-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:45:22.420685 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db54c63-b876-494f-87d2-e09fa020ad59 tempest-VolumeQuotaClassesTest-2013247953 tempest-VolumeQuotaClassesTest-2013247953-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:22.422318 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db54c63-b876-494f-87d2-e09fa020ad59 tempest-VolumeQuotaClassesTest-2013247953 tempest-VolumeQuotaClassesTest-2013247953-project-admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:22.437328 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 745/1481] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 08:45:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:45:22.477066 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26551dcc-f813-4413-a4f3-ab7641e7a8d6 tempest-VolumeQuotaClassesTest-2013247953 tempest-VolumeQuotaClassesTest-2013247953-project-admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:22.487278 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26551dcc-f813-4413-a4f3-ab7641e7a8d6 tempest-VolumeQuotaClassesTest-2013247953 tempest-VolumeQuotaClassesTest-2013247953-project-admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:22.491843 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f351e9f-8415-474c-93e0-9941cfbcae8b tempest-VolumeQuotaClassesTest-2013247953 tempest-VolumeQuotaClassesTest-2013247953-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbf2d6e9a9b14ccb852f0881d7ad8e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c44dfc9d2a9488894c53429f2250026', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:22.492465 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f351e9f-8415-474c-93e0-9941cfbcae8b tempest-VolumeQuotaClassesTest-2013247953 tempest-VolumeQuotaClassesTest-2013247953-project-admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:22.492750 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f351e9f-8415-474c-93e0-9941cfbcae8b tempest-VolumeQuotaClassesTest-2013247953 tempest-VolumeQuotaClassesTest-2013247953-project-admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:22.493028 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f351e9f-8415-474c-93e0-9941cfbcae8b tempest-VolumeQuotaClassesTest-2013247953 tempest-VolumeQuotaClassesTest-2013247953-project-admin] PATH_INFO: `/v3/projects/80994aaacc754444b809fb815caa2e15` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:22.494158 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f351e9f-8415-474c-93e0-9941cfbcae8b tempest-VolumeQuotaClassesTest-2013247953 tempest-VolumeQuotaClassesTest-2013247953-project-admin] RBAC: Authorizing `identity:delete_project(project_id=80994aaacc754444b809fb815caa2e15)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:22.496068 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f351e9f-8415-474c-93e0-9941cfbcae8b tempest-VolumeQuotaClassesTest-2013247953 tempest-VolumeQuotaClassesTest-2013247953-project-admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:22.542260 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-6f351e9f-8415-474c-93e0-9941cfbcae8b tempest-VolumeQuotaClassesTest-2013247953 tempest-VolumeQuotaClassesTest-2013247953-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80994aaacc754444b809fb815caa2e15', 'request_id': 'req-6f351e9f-8415-474c-93e0-9941cfbcae8b'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:22.548793 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-6f351e9f-8415-474c-93e0-9941cfbcae8b tempest-VolumeQuotaClassesTest-2013247953 tempest-VolumeQuotaClassesTest-2013247953-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80994aaacc754444b809fb815caa2e15', 'request_id': 'req-6f351e9f-8415-474c-93e0-9941cfbcae8b'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:22.556069 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-6f351e9f-8415-474c-93e0-9941cfbcae8b tempest-VolumeQuotaClassesTest-2013247953 tempest-VolumeQuotaClassesTest-2013247953-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80994aaacc754444b809fb815caa2e15', 'request_id': 'req-6f351e9f-8415-474c-93e0-9941cfbcae8b'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:22.566469 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-6f351e9f-8415-474c-93e0-9941cfbcae8b tempest-VolumeQuotaClassesTest-2013247953 tempest-VolumeQuotaClassesTest-2013247953-project-admin] The token cache is being invalidate because project 80994aaacc754444b809fb815caa2e15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 08:45:22.566756 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-6f351e9f-8415-474c-93e0-9941cfbcae8b tempest-VolumeQuotaClassesTest-2013247953 tempest-VolumeQuotaClassesTest-2013247953-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:22.567398 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-6f351e9f-8415-474c-93e0-9941cfbcae8b tempest-VolumeQuotaClassesTest-2013247953 tempest-VolumeQuotaClassesTest-2013247953-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:22.568883 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 737/1482] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 08:45:22 2022] DELETE /identity/v3/projects/80994aaacc754444b809fb815caa2e15 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:22.779632 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db54c63-b876-494f-87d2-e09fa020ad59 tempest-VolumeQuotaClassesTest-2013247953 tempest-VolumeQuotaClassesTest-2013247953-project-admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:22.781527 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-3db54c63-b876-494f-87d2-e09fa020ad59 tempest-VolumeQuotaClassesTest-2013247953 tempest-VolumeQuotaClassesTest-2013247953-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:22.813265 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:22.850634 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db54c63-b876-494f-87d2-e09fa020ad59 tempest-VolumeQuotaClassesTest-2013247953 tempest-VolumeQuotaClassesTest-2013247953-project-admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:22.915397 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8269b81d-ceec-49e6-9296-1d3af7c1ce23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:22.916044 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269b81d-ceec-49e6-9296-1d3af7c1ce23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:22.916335 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269b81d-ceec-49e6-9296-1d3af7c1ce23 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:22.916614 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269b81d-ceec-49e6-9296-1d3af7c1ce23 admin admin] PATH_INFO: `/v3/users/57a48343bb0e470b8657e81c4e990f1c` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:22.917825 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8269b81d-ceec-49e6-9296-1d3af7c1ce23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57a48343bb0e470b8657e81c4e990f1c)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:22.919751 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8269b81d-ceec-49e6-9296-1d3af7c1ce23 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:22.967790 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8269b81d-ceec-49e6-9296-1d3af7c1ce23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57a48343bb0e470b8657e81c4e990f1c'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:22.968510 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8269b81d-ceec-49e6-9296-1d3af7c1ce23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57a48343bb0e470b8657e81c4e990f1c'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:22.975510 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8269b81d-ceec-49e6-9296-1d3af7c1ce23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57a48343bb0e470b8657e81c4e990f1c'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:22.988980 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8269b81d-ceec-49e6-9296-1d3af7c1ce23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57a48343bb0e470b8657e81c4e990f1c'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:22.997415 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8269b81d-ceec-49e6-9296-1d3af7c1ce23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57a48343bb0e470b8657e81c4e990f1c'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:23.004695 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 746/1483] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 08:45:22 2022] DELETE /identity/v3/users/57a48343bb0e470b8657e81c4e990f1c => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:23.009652 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc399b-b025-4765-bc08-ac637eb894ca None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:23.010130 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc399b-b025-4765-bc08-ac637eb894ca None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:23.010255 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc399b-b025-4765-bc08-ac637eb894ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:23.030736 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-a0fc399b-b025-4765-bc08-ac637eb894ca None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:23.032440 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-a0fc399b-b025-4765-bc08-ac637eb894ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:23.099359 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:23.118806 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 738/1484] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:45:23.130115 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8269b81d-ceec-49e6-9296-1d3af7c1ce23 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:23.131601 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-8269b81d-ceec-49e6-9296-1d3af7c1ce23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:23.211695 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8269b81d-ceec-49e6-9296-1d3af7c1ce23 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:23.215478 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c6339d-70ab-4802-b582-2892e0f6fa39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:23.216094 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6339d-70ab-4802-b582-2892e0f6fa39 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:23.216378 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6339d-70ab-4802-b582-2892e0f6fa39 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:23.216680 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6339d-70ab-4802-b582-2892e0f6fa39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:23.223995 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:23.227243 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c6339d-70ab-4802-b582-2892e0f6fa39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:23.229330 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c6339d-70ab-4802-b582-2892e0f6fa39 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:23.243113 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 747/1485] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:45:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:45:23.406330 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e90132a-5160-480c-aa47-e7b5a6459d63 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:23.406629 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e90132a-5160-480c-aa47-e7b5a6459d63 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:23.406980 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e90132a-5160-480c-aa47-e7b5a6459d63 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:23.408416 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 739/1486] 10.222.0.30 () {54 vars in 921 bytes} [Sat Oct 15 08:45:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:45:23.421964 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab507a-ba90-4369-93e1-d6178e55ded4 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:23.422455 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab507a-ba90-4369-93e1-d6178e55ded4 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:23.422455 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab507a-ba90-4369-93e1-d6178e55ded4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:23.463366 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-4cab507a-ba90-4369-93e1-d6178e55ded4 None None] MFA Rules not processed for user `af0a8af5cadc4cc1ab73e8eb2a20bb7e`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:23.464620 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-4cab507a-ba90-4369-93e1-d6178e55ded4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:23.618806 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 748/1487] 10.222.0.30 () {58 vars in 1018 bytes} [Sat Oct 15 08:45:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:45:23.636640 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e90132a-5160-480c-aa47-e7b5a6459d63 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:23.640356 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-2e90132a-5160-480c-aa47-e7b5a6459d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:23.686406 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e90132a-5160-480c-aa47-e7b5a6459d63 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:23.689156 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cab507a-ba90-4369-93e1-d6178e55ded4 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:23.691215 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-4cab507a-ba90-4369-93e1-d6178e55ded4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:23.720504 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab99ae4-af28-40a7-9f8b-27f1711a4814 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0a8af5cadc4cc1ab73e8eb2a20bb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:23.721154 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab99ae4-af28-40a7-9f8b-27f1711a4814 service nova] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:23.722204 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cab507a-ba90-4369-93e1-d6178e55ded4 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:23.723054 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab99ae4-af28-40a7-9f8b-27f1711a4814 service nova] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:23.723231 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab99ae4-af28-40a7-9f8b-27f1711a4814 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:23.736120 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126c3716-b886-474d-a6f3-dc657300aba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:23.736365 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c3716-b886-474d-a6f3-dc657300aba8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:23.736708 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c3716-b886-474d-a6f3-dc657300aba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:23.737019 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c3716-b886-474d-a6f3-dc657300aba8 admin admin] PATH_INFO: `/v3/projects/ac4e88c95eab423c90afec990252a701` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:23.739327 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126c3716-b886-474d-a6f3-dc657300aba8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac4e88c95eab423c90afec990252a701)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:23.742977 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126c3716-b886-474d-a6f3-dc657300aba8 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:23.745794 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab99ae4-af28-40a7-9f8b-27f1711a4814 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:23.747413 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab99ae4-af28-40a7-9f8b-27f1711a4814 service nova] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:23.773653 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 740/1488] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:45:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:45:23.808244 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-126c3716-b886-474d-a6f3-dc657300aba8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac4e88c95eab423c90afec990252a701', 'request_id': 'req-126c3716-b886-474d-a6f3-dc657300aba8'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:23.814851 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-126c3716-b886-474d-a6f3-dc657300aba8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac4e88c95eab423c90afec990252a701', 'request_id': 'req-126c3716-b886-474d-a6f3-dc657300aba8'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:23.824896 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-126c3716-b886-474d-a6f3-dc657300aba8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac4e88c95eab423c90afec990252a701', 'request_id': 'req-126c3716-b886-474d-a6f3-dc657300aba8'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:23.837758 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-126c3716-b886-474d-a6f3-dc657300aba8 admin admin] The token cache is being invalidate because project ac4e88c95eab423c90afec990252a701 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 08:45:23.838056 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-126c3716-b886-474d-a6f3-dc657300aba8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:23.838721 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-126c3716-b886-474d-a6f3-dc657300aba8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:23.840250 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 749/1489] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 08:45:23 2022] DELETE /identity/v3/projects/ac4e88c95eab423c90afec990252a701 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 08:45:23.844366 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab99ae4-af28-40a7-9f8b-27f1711a4814 service nova] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:23.855560 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab99ae4-af28-40a7-9f8b-27f1711a4814 service nova] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:23.915846 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98c1e5a-e243-4265-9e6d-45fb2a790c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:23.916719 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98c1e5a-e243-4265-9e6d-45fb2a790c36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:23.917257 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98c1e5a-e243-4265-9e6d-45fb2a790c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:23.917656 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98c1e5a-e243-4265-9e6d-45fb2a790c36 admin admin] PATH_INFO: `/v3/users/cbf2d6e9a9b14ccb852f0881d7ad8e31` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:23.919079 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98c1e5a-e243-4265-9e6d-45fb2a790c36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbf2d6e9a9b14ccb852f0881d7ad8e31)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:23.921254 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98c1e5a-e243-4265-9e6d-45fb2a790c36 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:23.973646 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-a98c1e5a-e243-4265-9e6d-45fb2a790c36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbf2d6e9a9b14ccb852f0881d7ad8e31'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:23.974583 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-a98c1e5a-e243-4265-9e6d-45fb2a790c36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbf2d6e9a9b14ccb852f0881d7ad8e31'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:23.990376 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-a98c1e5a-e243-4265-9e6d-45fb2a790c36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbf2d6e9a9b14ccb852f0881d7ad8e31'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:24.000110 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-a98c1e5a-e243-4265-9e6d-45fb2a790c36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbf2d6e9a9b14ccb852f0881d7ad8e31'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:24.000767 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-a98c1e5a-e243-4265-9e6d-45fb2a790c36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbf2d6e9a9b14ccb852f0881d7ad8e31'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:24.038234 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 741/1490] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 08:45:23 2022] DELETE /identity/v3/users/cbf2d6e9a9b14ccb852f0881d7ad8e31 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:24.344233 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126c3716-b886-474d-a6f3-dc657300aba8 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:24.345637 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-126c3716-b886-474d-a6f3-dc657300aba8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:24.364028 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126c3716-b886-474d-a6f3-dc657300aba8 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:24.424745 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d9b0e1-6bd2-460d-8919-f16a9ab264d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:24.425377 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d9b0e1-6bd2-460d-8919-f16a9ab264d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:24.425590 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d9b0e1-6bd2-460d-8919-f16a9ab264d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:24.425906 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d9b0e1-6bd2-460d-8919-f16a9ab264d0 admin admin] PATH_INFO: `/v3/projects/6c44dfc9d2a9488894c53429f2250026` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:24.427144 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d9b0e1-6bd2-460d-8919-f16a9ab264d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c44dfc9d2a9488894c53429f2250026)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:24.429667 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d9b0e1-6bd2-460d-8919-f16a9ab264d0 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:24.441444 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:24.479325 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-60d9b0e1-6bd2-460d-8919-f16a9ab264d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c44dfc9d2a9488894c53429f2250026', 'request_id': 'req-60d9b0e1-6bd2-460d-8919-f16a9ab264d0'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:24.486020 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-60d9b0e1-6bd2-460d-8919-f16a9ab264d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c44dfc9d2a9488894c53429f2250026', 'request_id': 'req-60d9b0e1-6bd2-460d-8919-f16a9ab264d0'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:24.494197 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-60d9b0e1-6bd2-460d-8919-f16a9ab264d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c44dfc9d2a9488894c53429f2250026', 'request_id': 'req-60d9b0e1-6bd2-460d-8919-f16a9ab264d0'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:24.506249 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-60d9b0e1-6bd2-460d-8919-f16a9ab264d0 admin admin] The token cache is being invalidate because project 6c44dfc9d2a9488894c53429f2250026 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 08:45:24.506568 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-60d9b0e1-6bd2-460d-8919-f16a9ab264d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:24.507216 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-60d9b0e1-6bd2-460d-8919-f16a9ab264d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:24.509699 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 750/1491] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 08:45:24 2022] DELETE /identity/v3/projects/6c44dfc9d2a9488894c53429f2250026 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:24.514059 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bbd135-51ac-4998-8c76-47493202e2e7 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:24.514351 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bbd135-51ac-4998-8c76-47493202e2e7 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:24.514633 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bbd135-51ac-4998-8c76-47493202e2e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:24.536351 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-58bbd135-51ac-4998-8c76-47493202e2e7 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:24.537596 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-58bbd135-51ac-4998-8c76-47493202e2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:24.603188 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:24.613317 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 742/1492] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 08:45:24.621535 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedefb04-cb0e-4c56-99de-ab8d18b89593 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:24.621865 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedefb04-cb0e-4c56-99de-ab8d18b89593 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:24.622146 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedefb04-cb0e-4c56-99de-ab8d18b89593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:24.641591 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-fedefb04-cb0e-4c56-99de-ab8d18b89593 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:24.642827 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-fedefb04-cb0e-4c56-99de-ab8d18b89593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:24.650949 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 751/1493] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:45:24.655199 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bbd135-51ac-4998-8c76-47493202e2e7 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:24.665320 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bbd135-51ac-4998-8c76-47493202e2e7 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:24.669916 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8605de45-88d1-444c-a214-e8870e5a083f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:24.670537 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8605de45-88d1-444c-a214-e8870e5a083f admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:24.670812 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8605de45-88d1-444c-a214-e8870e5a083f admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:24.671084 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8605de45-88d1-444c-a214-e8870e5a083f admin admin] PATH_INFO: `/v3/domains` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:24.671725 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8605de45-88d1-444c-a214-e8870e5a083f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:45:24.672026 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8605de45-88d1-444c-a214-e8870e5a083f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:24.673616 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8605de45-88d1-444c-a214-e8870e5a083f admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:24.685497 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 743/1494] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 08:45:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:45:24.689799 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedefb04-cb0e-4c56-99de-ab8d18b89593 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:24.699169 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedefb04-cb0e-4c56-99de-ab8d18b89593 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:24.703784 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e805f132-368f-484b-b42a-d7bdb8a64863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:24.704377 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e805f132-368f-484b-b42a-d7bdb8a64863 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:24.704657 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e805f132-368f-484b-b42a-d7bdb8a64863 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:24.704933 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e805f132-368f-484b-b42a-d7bdb8a64863 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:24.705565 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e805f132-368f-484b-b42a-d7bdb8a64863 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:24.707488 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e805f132-368f-484b-b42a-d7bdb8a64863 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:24.737135 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 752/1495] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 08:45:24 2022] POST /identity/v3/projects => generated 391 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:45:24.738255 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8605de45-88d1-444c-a214-e8870e5a083f admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:24.748401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8605de45-88d1-444c-a214-e8870e5a083f admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:24.805172 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa5d34c-8fa9-491d-bd6d-5192e7999a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:24.805824 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa5d34c-8fa9-491d-bd6d-5192e7999a9c admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:24.806108 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa5d34c-8fa9-491d-bd6d-5192e7999a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:24.806383 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa5d34c-8fa9-491d-bd6d-5192e7999a9c admin admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:24.807004 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa5d34c-8fa9-491d-bd6d-5192e7999a9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:24.808912 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa5d34c-8fa9-491d-bd6d-5192e7999a9c admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:24.830158 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 744/1496] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:45:24 2022] POST /identity/v3/users => generated 296 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 08:45:24.834297 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e805f132-368f-484b-b42a-d7bdb8a64863 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:24.844750 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e805f132-368f-484b-b42a-d7bdb8a64863 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:24.849249 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab330826-1667-41d9-83e1-a8600c1f4e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:24.849891 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab330826-1667-41d9-83e1-a8600c1f4e07 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:24.850178 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab330826-1667-41d9-83e1-a8600c1f4e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:24.850451 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab330826-1667-41d9-83e1-a8600c1f4e07 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:24.851625 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab330826-1667-41d9-83e1-a8600c1f4e07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:24.853211 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab330826-1667-41d9-83e1-a8600c1f4e07 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:24.863288 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 753/1497] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:45:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:45:24.867878 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa5d34c-8fa9-491d-bd6d-5192e7999a9c admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:24.877546 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa5d34c-8fa9-491d-bd6d-5192e7999a9c admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:24.882034 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51227e25-5f84-458a-bcdd-6017534bfac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:24.882627 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51227e25-5f84-458a-bcdd-6017534bfac9 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:24.882907 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51227e25-5f84-458a-bcdd-6017534bfac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:24.883179 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51227e25-5f84-458a-bcdd-6017534bfac9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:24.883809 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51227e25-5f84-458a-bcdd-6017534bfac9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:24.885399 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51227e25-5f84-458a-bcdd-6017534bfac9 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:24.895740 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 745/1498] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:45:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:45:24.900109 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab330826-1667-41d9-83e1-a8600c1f4e07 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:24.909955 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab330826-1667-41d9-83e1-a8600c1f4e07 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:24.915014 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf0f18-2bb4-446b-a300-b4d8a4e9c6b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:24.915659 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf0f18-2bb4-446b-a300-b4d8a4e9c6b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:24.915942 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf0f18-2bb4-446b-a300-b4d8a4e9c6b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:24.916218 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf0f18-2bb4-446b-a300-b4d8a4e9c6b2 admin admin] PATH_INFO: `/v3/projects/b10114bd18b443e79c4d5426f7100418/users/2f7f8070839e46a88a43dde5d35efbac/roles/3248f0268738489b883041851d681bb7` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:24.932035 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdf0f18-2bb4-446b-a300-b4d8a4e9c6b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b10114bd18b443e79c4d5426f7100418, user_id=2f7f8070839e46a88a43dde5d35efbac, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:24.934700 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdf0f18-2bb4-446b-a300-b4d8a4e9c6b2 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:24.948456 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 754/1499] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:45:24 2022] PUT /identity/v3/projects/b10114bd18b443e79c4d5426f7100418/users/2f7f8070839e46a88a43dde5d35efbac/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:24.961320 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b572d7-3bb6-470e-8fec-7464a58a3739 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:24.961593 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b572d7-3bb6-470e-8fec-7464a58a3739 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:24.961922 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b572d7-3bb6-470e-8fec-7464a58a3739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:24.992711 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-40b572d7-3bb6-470e-8fec-7464a58a3739 None None] MFA Rules not processed for user `2f7f8070839e46a88a43dde5d35efbac`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:24.994084 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-40b572d7-3bb6-470e-8fec-7464a58a3739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:25.061740 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 746/1500] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:24 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:45:25.066350 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf0f18-2bb4-446b-a300-b4d8a4e9c6b2 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:25.138263 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf0f18-2bb4-446b-a300-b4d8a4e9c6b2 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:25.142999 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55674009-efe9-473e-a106-0b469fee2fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:25.143616 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55674009-efe9-473e-a106-0b469fee2fd6 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:25.143959 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55674009-efe9-473e-a106-0b469fee2fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:25.144241 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55674009-efe9-473e-a106-0b469fee2fd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:25.144892 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55674009-efe9-473e-a106-0b469fee2fd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:25.146919 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55674009-efe9-473e-a106-0b469fee2fd6 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:25.180110 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 755/1501] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 08:45:25 2022] POST /identity/v3/projects => generated 391 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:45:25.181190 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b572d7-3bb6-470e-8fec-7464a58a3739 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:25.199438 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b572d7-3bb6-470e-8fec-7464a58a3739 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:25.251279 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:25.260384 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365c682d-e3f8-4936-84b8-5445815ae22f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:25.261226 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365c682d-e3f8-4936-84b8-5445815ae22f admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:25.261662 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365c682d-e3f8-4936-84b8-5445815ae22f admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:25.262069 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365c682d-e3f8-4936-84b8-5445815ae22f admin admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:25.262723 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365c682d-e3f8-4936-84b8-5445815ae22f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:25.264647 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365c682d-e3f8-4936-84b8-5445815ae22f admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:25.287319 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 747/1502] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:45:25 2022] POST /identity/v3/users => generated 302 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:45:25.291308 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55674009-efe9-473e-a106-0b469fee2fd6 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:25.301574 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55674009-efe9-473e-a106-0b469fee2fd6 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:25.306844 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2babb3ef-b49f-4cb6-9a22-1817004a46a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:25.307438 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2babb3ef-b49f-4cb6-9a22-1817004a46a1 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:25.307718 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2babb3ef-b49f-4cb6-9a22-1817004a46a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:25.307988 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2babb3ef-b49f-4cb6-9a22-1817004a46a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:25.308620 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2babb3ef-b49f-4cb6-9a22-1817004a46a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:25.310231 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2babb3ef-b49f-4cb6-9a22-1817004a46a1 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:25.322290 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 756/1503] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:45:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:45:25.326238 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365c682d-e3f8-4936-84b8-5445815ae22f admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:25.337935 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365c682d-e3f8-4936-84b8-5445815ae22f admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:25.343658 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0432cd-020e-456b-8ec9-d18abaaac824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:25.344472 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0432cd-020e-456b-8ec9-d18abaaac824 admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:25.344835 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0432cd-020e-456b-8ec9-d18abaaac824 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:25.345177 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0432cd-020e-456b-8ec9-d18abaaac824 admin admin] PATH_INFO: `/v3/projects/80009ae6678542519a5dfb76cdcc8f57/users/2c4b78f0ce07437487095f330ce0d26a/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:25.362084 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0432cd-020e-456b-8ec9-d18abaaac824 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80009ae6678542519a5dfb76cdcc8f57, user_id=2c4b78f0ce07437487095f330ce0d26a, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:25.364965 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0432cd-020e-456b-8ec9-d18abaaac824 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:25.378586 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 748/1504] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:45:25 2022] PUT /identity/v3/projects/80009ae6678542519a5dfb76cdcc8f57/users/2c4b78f0ce07437487095f330ce0d26a/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 08:45:25.382705 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2babb3ef-b49f-4cb6-9a22-1817004a46a1 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:25.393306 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2babb3ef-b49f-4cb6-9a22-1817004a46a1 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:25.406963 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:25.460343 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cf12e8-4539-47d2-a29c-1190d169ba78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:25.460959 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cf12e8-4539-47d2-a29c-1190d169ba78 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:25.461106 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cf12e8-4539-47d2-a29c-1190d169ba78 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:25.461379 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cf12e8-4539-47d2-a29c-1190d169ba78 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:25.462049 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cf12e8-4539-47d2-a29c-1190d169ba78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:25.463636 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cf12e8-4539-47d2-a29c-1190d169ba78 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:25.477624 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 757/1505] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:45:25 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:45:25.479046 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0432cd-020e-456b-8ec9-d18abaaac824 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:25.489414 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0432cd-020e-456b-8ec9-d18abaaac824 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:25.494617 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792c144b-39ff-4093-b178-4ad55f7417d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:25.495272 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c144b-39ff-4093-b178-4ad55f7417d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:25.495561 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c144b-39ff-4093-b178-4ad55f7417d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:25.495834 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c144b-39ff-4093-b178-4ad55f7417d9 admin admin] PATH_INFO: `/v3/projects/80009ae6678542519a5dfb76cdcc8f57/users/2c4b78f0ce07437487095f330ce0d26a/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:25.498047 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792c144b-39ff-4093-b178-4ad55f7417d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80009ae6678542519a5dfb76cdcc8f57, user_id=2c4b78f0ce07437487095f330ce0d26a, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:25.500695 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792c144b-39ff-4093-b178-4ad55f7417d9 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:25.511657 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 749/1506] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:45:25 2022] PUT /identity/v3/projects/80009ae6678542519a5dfb76cdcc8f57/users/2c4b78f0ce07437487095f330ce0d26a/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:25.522049 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c4b0f0-bbf3-4cff-a938-76c2a98932db None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:25.522402 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c4b0f0-bbf3-4cff-a938-76c2a98932db None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:25.522747 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c4b0f0-bbf3-4cff-a938-76c2a98932db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:25.554671 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-b7c4b0f0-bbf3-4cff-a938-76c2a98932db None None] MFA Rules not processed for user `2c4b78f0ce07437487095f330ce0d26a`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:25.555879 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-b7c4b0f0-bbf3-4cff-a938-76c2a98932db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:25.623796 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 758/1507] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:25 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:45:25.634493 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792c144b-39ff-4093-b178-4ad55f7417d9 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:25.636378 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-792c144b-39ff-4093-b178-4ad55f7417d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:25.700171 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792c144b-39ff-4093-b178-4ad55f7417d9 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:25.705109 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09097308-c828-4959-9687-3250655b8678 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:25.705793 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09097308-c828-4959-9687-3250655b8678 service cinder] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:25.706407 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09097308-c828-4959-9687-3250655b8678 service cinder] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:25.706619 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09097308-c828-4959-9687-3250655b8678 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:25.719326 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09097308-c828-4959-9687-3250655b8678 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:25.720690 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09097308-c828-4959-9687-3250655b8678 service cinder] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:25.732652 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 750/1508] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:45:25 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:45:25.790594 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c4b0f0-bbf3-4cff-a938-76c2a98932db None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:25.791985 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-b7c4b0f0-bbf3-4cff-a938-76c2a98932db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:25.804027 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c4b0f0-bbf3-4cff-a938-76c2a98932db None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:25.814280 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fb8375-3d03-4fbb-8048-d6a4dff16607 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:25.814914 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fb8375-3d03-4fbb-8048-d6a4dff16607 service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:25.815198 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fb8375-3d03-4fbb-8048-d6a4dff16607 service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:25.815481 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fb8375-3d03-4fbb-8048-d6a4dff16607 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:25.893044 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fb8375-3d03-4fbb-8048-d6a4dff16607 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:25.894575 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fb8375-3d03-4fbb-8048-d6a4dff16607 service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:25.904642 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 759/1509] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:45:25 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:45:31.652789 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09097308-c828-4959-9687-3250655b8678 service cinder] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:31.654424 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-09097308-c828-4959-9687-3250655b8678 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:31.732773 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09097308-c828-4959-9687-3250655b8678 service cinder] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:31.737313 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4e187a-ee8e-48e7-8e09-da4e8cbd3b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:31.738093 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e187a-ee8e-48e7-8e09-da4e8cbd3b90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:31.738417 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e187a-ee8e-48e7-8e09-da4e8cbd3b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:31.738773 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e187a-ee8e-48e7-8e09-da4e8cbd3b90 admin admin] PATH_INFO: `/v3/users/344016e60c0543a4a2494c4c239287dd` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:31.740087 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4e187a-ee8e-48e7-8e09-da4e8cbd3b90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=344016e60c0543a4a2494c4c239287dd)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:31.742189 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4e187a-ee8e-48e7-8e09-da4e8cbd3b90 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:31.786115 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fb8375-3d03-4fbb-8048-d6a4dff16607 service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:31.797545 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fb8375-3d03-4fbb-8048-d6a4dff16607 service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:31.800933 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-fd4e187a-ee8e-48e7-8e09-da4e8cbd3b90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '344016e60c0543a4a2494c4c239287dd'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:31.801632 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-fd4e187a-ee8e-48e7-8e09-da4e8cbd3b90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '344016e60c0543a4a2494c4c239287dd'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:31.802590 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c144a3af-901a-4ad8-a408-44d80d55b8ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:31.803247 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c144a3af-901a-4ad8-a408-44d80d55b8ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:31.803529 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c144a3af-901a-4ad8-a408-44d80d55b8ae admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:31.803809 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c144a3af-901a-4ad8-a408-44d80d55b8ae admin admin] PATH_INFO: `/v3/users/2f7f8070839e46a88a43dde5d35efbac` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:31.805062 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c144a3af-901a-4ad8-a408-44d80d55b8ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f7f8070839e46a88a43dde5d35efbac)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:31.807091 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c144a3af-901a-4ad8-a408-44d80d55b8ae admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:31.807668 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:31.818563 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-fd4e187a-ee8e-48e7-8e09-da4e8cbd3b90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '344016e60c0543a4a2494c4c239287dd'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:31.829087 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-fd4e187a-ee8e-48e7-8e09-da4e8cbd3b90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '344016e60c0543a4a2494c4c239287dd'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:31.829812 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-fd4e187a-ee8e-48e7-8e09-da4e8cbd3b90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '344016e60c0543a4a2494c4c239287dd'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:31.841098 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:31.847545 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 751/1510] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 08:45:31 2022] DELETE /identity/v3/users/344016e60c0543a4a2494c4c239287dd => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:31.852476 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559f9eb9-ddea-472d-a919-f367b5c37e4a None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:31.853009 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559f9eb9-ddea-472d-a919-f367b5c37e4a None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:31.854203 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559f9eb9-ddea-472d-a919-f367b5c37e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:31.867941 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-c144a3af-901a-4ad8-a408-44d80d55b8ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f7f8070839e46a88a43dde5d35efbac'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:31.868650 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-c144a3af-901a-4ad8-a408-44d80d55b8ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f7f8070839e46a88a43dde5d35efbac'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:31.876330 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-c144a3af-901a-4ad8-a408-44d80d55b8ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f7f8070839e46a88a43dde5d35efbac'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:31.876709 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-559f9eb9-ddea-472d-a919-f367b5c37e4a None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:31.878292 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-559f9eb9-ddea-472d-a919-f367b5c37e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:31.889757 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-c144a3af-901a-4ad8-a408-44d80d55b8ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f7f8070839e46a88a43dde5d35efbac'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:31.898796 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-c144a3af-901a-4ad8-a408-44d80d55b8ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f7f8070839e46a88a43dde5d35efbac'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:31.906136 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 760/1511] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 08:45:31 2022] DELETE /identity/v3/users/2f7f8070839e46a88a43dde5d35efbac => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 08:45:31.910462 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e49e732-56a8-4fce-b237-e15c9bac1f78 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:31.910820 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e49e732-56a8-4fce-b237-e15c9bac1f78 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:31.911099 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e49e732-56a8-4fce-b237-e15c9bac1f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:31.938255 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-1e49e732-56a8-4fce-b237-e15c9bac1f78 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:31.939489 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-1e49e732-56a8-4fce-b237-e15c9bac1f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:31.947873 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 752/1512] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 08:45:31.959426 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559f9eb9-ddea-472d-a919-f367b5c37e4a None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:31.962245 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-559f9eb9-ddea-472d-a919-f367b5c37e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:32.028940 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 761/1513] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 08:45:32.035317 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559f9eb9-ddea-472d-a919-f367b5c37e4a None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:32.040739 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e49e732-56a8-4fce-b237-e15c9bac1f78 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:32.119567 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dad1d0-3f15-48dd-abc4-a56da027101c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:32.120038 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dad1d0-3f15-48dd-abc4-a56da027101c service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:32.120405 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dad1d0-3f15-48dd-abc4-a56da027101c service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:32.120601 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dad1d0-3f15-48dd-abc4-a56da027101c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:32.137273 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dad1d0-3f15-48dd-abc4-a56da027101c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:32.139282 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dad1d0-3f15-48dd-abc4-a56da027101c service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:32.142542 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e49e732-56a8-4fce-b237-e15c9bac1f78 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:32.150254 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f733e2-fe3f-4a92-b28e-144173530078 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:32.151423 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f733e2-fe3f-4a92-b28e-144173530078 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:32.152047 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f733e2-fe3f-4a92-b28e-144173530078 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:32.152462 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f733e2-fe3f-4a92-b28e-144173530078 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:32.153724 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 753/1514] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:45:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 08:45:32.172342 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f733e2-fe3f-4a92-b28e-144173530078 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:32.175068 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f733e2-fe3f-4a92-b28e-144173530078 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:32.188409 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 762/1515] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:45:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 08:45:32.864038 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dad1d0-3f15-48dd-abc4-a56da027101c service neutron] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:32.865610 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-23dad1d0-3f15-48dd-abc4-a56da027101c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:32.871287 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f733e2-fe3f-4a92-b28e-144173530078 service neutron] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:32.873033 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-d9f733e2-fe3f-4a92-b28e-144173530078 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:32.899528 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f733e2-fe3f-4a92-b28e-144173530078 service neutron] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:32.903422 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dad1d0-3f15-48dd-abc4-a56da027101c service neutron] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:32.911558 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fe8cff-bbf5-4ea4-9c11-263ba4c96f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:32.912968 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe8cff-bbf5-4ea4-9c11-263ba4c96f28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:32.913766 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7120deda-a734-4129-93fc-89ca64fd4fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:32.914604 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7120deda-a734-4129-93fc-89ca64fd4fd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:32.915106 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7120deda-a734-4129-93fc-89ca64fd4fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:32.915586 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7120deda-a734-4129-93fc-89ca64fd4fd3 admin admin] PATH_INFO: `/v3/projects/b10114bd18b443e79c4d5426f7100418` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:32.915692 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe8cff-bbf5-4ea4-9c11-263ba4c96f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:32.915972 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe8cff-bbf5-4ea4-9c11-263ba4c96f28 admin admin] PATH_INFO: `/v3/projects/7fd58de14fda4ec8bb3c59e0938fac20` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:32.917331 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fe8cff-bbf5-4ea4-9c11-263ba4c96f28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fd58de14fda4ec8bb3c59e0938fac20)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:32.917942 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7120deda-a734-4129-93fc-89ca64fd4fd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b10114bd18b443e79c4d5426f7100418)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:32.919879 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fe8cff-bbf5-4ea4-9c11-263ba4c96f28 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:32.920443 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7120deda-a734-4129-93fc-89ca64fd4fd3 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:32.959484 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:32.979161 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:32.991168 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-18fe8cff-bbf5-4ea4-9c11-263ba4c96f28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fd58de14fda4ec8bb3c59e0938fac20', 'request_id': 'req-18fe8cff-bbf5-4ea4-9c11-263ba4c96f28'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:32.992034 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7120deda-a734-4129-93fc-89ca64fd4fd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b10114bd18b443e79c4d5426f7100418', 'request_id': 'req-7120deda-a734-4129-93fc-89ca64fd4fd3'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:33.000026 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7120deda-a734-4129-93fc-89ca64fd4fd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b10114bd18b443e79c4d5426f7100418', 'request_id': 'req-7120deda-a734-4129-93fc-89ca64fd4fd3'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:33.000463 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-18fe8cff-bbf5-4ea4-9c11-263ba4c96f28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fd58de14fda4ec8bb3c59e0938fac20', 'request_id': 'req-18fe8cff-bbf5-4ea4-9c11-263ba4c96f28'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:33.009946 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7120deda-a734-4129-93fc-89ca64fd4fd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b10114bd18b443e79c4d5426f7100418', 'request_id': 'req-7120deda-a734-4129-93fc-89ca64fd4fd3'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:33.014064 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-18fe8cff-bbf5-4ea4-9c11-263ba4c96f28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fd58de14fda4ec8bb3c59e0938fac20', 'request_id': 'req-18fe8cff-bbf5-4ea4-9c11-263ba4c96f28'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:33.022051 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7120deda-a734-4129-93fc-89ca64fd4fd3 admin admin] The token cache is being invalidate because project b10114bd18b443e79c4d5426f7100418 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 08:45:33.022687 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7120deda-a734-4129-93fc-89ca64fd4fd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:33.023711 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7120deda-a734-4129-93fc-89ca64fd4fd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:33.026162 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 763/1516] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 08:45:32 2022] DELETE /identity/v3/projects/b10114bd18b443e79c4d5426f7100418 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:33.029766 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7120deda-a734-4129-93fc-89ca64fd4fd3 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:33.031315 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-7120deda-a734-4129-93fc-89ca64fd4fd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:33.032511 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-18fe8cff-bbf5-4ea4-9c11-263ba4c96f28 admin admin] The token cache is being invalidate because project 7fd58de14fda4ec8bb3c59e0938fac20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 08:45:33.032733 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-18fe8cff-bbf5-4ea4-9c11-263ba4c96f28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:33.033538 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-18fe8cff-bbf5-4ea4-9c11-263ba4c96f28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:33.035232 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 754/1517] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 08:45:32 2022] DELETE /identity/v3/projects/7fd58de14fda4ec8bb3c59e0938fac20 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 08:45:33.039896 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fe8cff-bbf5-4ea4-9c11-263ba4c96f28 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:33.041490 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-18fe8cff-bbf5-4ea4-9c11-263ba4c96f28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:33.131744 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7120deda-a734-4129-93fc-89ca64fd4fd3 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:33.158810 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fe8cff-bbf5-4ea4-9c11-263ba4c96f28 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:33.164894 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d88d66-a5ff-4acc-aa79-8e86c32403d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:33.166159 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d88d66-a5ff-4acc-aa79-8e86c32403d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:33.166761 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d88d66-a5ff-4acc-aa79-8e86c32403d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:33.167192 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d88d66-a5ff-4acc-aa79-8e86c32403d1 admin admin] PATH_INFO: `/v3/users/1fe755f137d34169bc2b28369aa88d1b` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:33.168906 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d88d66-a5ff-4acc-aa79-8e86c32403d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fe755f137d34169bc2b28369aa88d1b)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:33.171527 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d88d66-a5ff-4acc-aa79-8e86c32403d1 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:33.238782 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcd604c-0e9a-424f-b860-99c650ed2aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:33.239766 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd604c-0e9a-424f-b860-99c650ed2aed admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:33.240148 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd604c-0e9a-424f-b860-99c650ed2aed admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:33.240586 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd604c-0e9a-424f-b860-99c650ed2aed admin admin] PATH_INFO: `/v3/users/2c4b78f0ce07437487095f330ce0d26a` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:33.242454 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcd604c-0e9a-424f-b860-99c650ed2aed admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c4b78f0ce07437487095f330ce0d26a)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:33.244771 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcd604c-0e9a-424f-b860-99c650ed2aed admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:33.245511 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-e5d88d66-a5ff-4acc-aa79-8e86c32403d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fe755f137d34169bc2b28369aa88d1b'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:33.246456 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-e5d88d66-a5ff-4acc-aa79-8e86c32403d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fe755f137d34169bc2b28369aa88d1b'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:33.265309 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-e5d88d66-a5ff-4acc-aa79-8e86c32403d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fe755f137d34169bc2b28369aa88d1b'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:33.278144 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-e5d88d66-a5ff-4acc-aa79-8e86c32403d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fe755f137d34169bc2b28369aa88d1b'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:33.279882 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-e5d88d66-a5ff-4acc-aa79-8e86c32403d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fe755f137d34169bc2b28369aa88d1b'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:33.318743 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 755/1518] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 08:45:33 2022] DELETE /identity/v3/users/1fe755f137d34169bc2b28369aa88d1b => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 08:45:33.329143 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-3bcd604c-0e9a-424f-b860-99c650ed2aed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c4b78f0ce07437487095f330ce0d26a'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:33.330003 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-3bcd604c-0e9a-424f-b860-99c650ed2aed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c4b78f0ce07437487095f330ce0d26a'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:33.339415 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-3bcd604c-0e9a-424f-b860-99c650ed2aed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c4b78f0ce07437487095f330ce0d26a'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:33.358017 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-3bcd604c-0e9a-424f-b860-99c650ed2aed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c4b78f0ce07437487095f330ce0d26a'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:33.365451 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:33.383224 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-3bcd604c-0e9a-424f-b860-99c650ed2aed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c4b78f0ce07437487095f330ce0d26a'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:33.395900 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 764/1519] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 08:45:33 2022] DELETE /identity/v3/users/2c4b78f0ce07437487095f330ce0d26a => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 08:45:33.960247 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d88d66-a5ff-4acc-aa79-8e86c32403d1 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:33.961396 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-e5d88d66-a5ff-4acc-aa79-8e86c32403d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:34.026690 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:34.092676 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcd604c-0e9a-424f-b860-99c650ed2aed admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:34.094979 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-3bcd604c-0e9a-424f-b860-99c650ed2aed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:34.132619 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d88d66-a5ff-4acc-aa79-8e86c32403d1 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:34.238887 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26b436e-711c-411d-8a20-52366c74d08b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:34.240949 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26b436e-711c-411d-8a20-52366c74d08b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:34.242164 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26b436e-711c-411d-8a20-52366c74d08b admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:34.243206 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26b436e-711c-411d-8a20-52366c74d08b admin admin] PATH_INFO: `/v3/projects/80009ae6678542519a5dfb76cdcc8f57` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:34.244987 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26b436e-711c-411d-8a20-52366c74d08b admin admin] RBAC: Authorizing `identity:delete_project(project_id=80009ae6678542519a5dfb76cdcc8f57)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:34.248693 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26b436e-711c-411d-8a20-52366c74d08b admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:34.251702 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcd604c-0e9a-424f-b860-99c650ed2aed admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:34.265859 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccbb6d2-2f0e-4870-94f5-e3b99f753a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:34.266761 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccbb6d2-2f0e-4870-94f5-e3b99f753a14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:34.266761 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccbb6d2-2f0e-4870-94f5-e3b99f753a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:34.266956 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccbb6d2-2f0e-4870-94f5-e3b99f753a14 admin admin] PATH_INFO: `/v3/projects/68baf8cd87e6498d93e7ddebe48a700e` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:34.268011 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccbb6d2-2f0e-4870-94f5-e3b99f753a14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68baf8cd87e6498d93e7ddebe48a700e)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:34.270325 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccbb6d2-2f0e-4870-94f5-e3b99f753a14 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:34.330384 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-a26b436e-711c-411d-8a20-52366c74d08b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80009ae6678542519a5dfb76cdcc8f57', 'request_id': 'req-a26b436e-711c-411d-8a20-52366c74d08b'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:34.351924 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-3ccbb6d2-2f0e-4870-94f5-e3b99f753a14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68baf8cd87e6498d93e7ddebe48a700e', 'request_id': 'req-3ccbb6d2-2f0e-4870-94f5-e3b99f753a14'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:34.354890 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-a26b436e-711c-411d-8a20-52366c74d08b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80009ae6678542519a5dfb76cdcc8f57', 'request_id': 'req-a26b436e-711c-411d-8a20-52366c74d08b'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:34.372255 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-3ccbb6d2-2f0e-4870-94f5-e3b99f753a14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68baf8cd87e6498d93e7ddebe48a700e', 'request_id': 'req-3ccbb6d2-2f0e-4870-94f5-e3b99f753a14'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:34.380902 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-3ccbb6d2-2f0e-4870-94f5-e3b99f753a14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68baf8cd87e6498d93e7ddebe48a700e', 'request_id': 'req-3ccbb6d2-2f0e-4870-94f5-e3b99f753a14'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:34.381397 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-a26b436e-711c-411d-8a20-52366c74d08b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80009ae6678542519a5dfb76cdcc8f57', 'request_id': 'req-a26b436e-711c-411d-8a20-52366c74d08b'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:34.403638 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-a26b436e-711c-411d-8a20-52366c74d08b admin admin] The token cache is being invalidate because project 80009ae6678542519a5dfb76cdcc8f57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 08:45:34.404690 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-a26b436e-711c-411d-8a20-52366c74d08b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:34.406136 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-a26b436e-711c-411d-8a20-52366c74d08b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:34.406338 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-3ccbb6d2-2f0e-4870-94f5-e3b99f753a14 admin admin] The token cache is being invalidate because project 68baf8cd87e6498d93e7ddebe48a700e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 08:45:34.406637 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-3ccbb6d2-2f0e-4870-94f5-e3b99f753a14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:34.407365 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-3ccbb6d2-2f0e-4870-94f5-e3b99f753a14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:34.408691 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 756/1520] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 08:45:33 2022] DELETE /identity/v3/projects/80009ae6678542519a5dfb76cdcc8f57 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:34.409087 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 765/1521] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 08:45:34 2022] DELETE /identity/v3/projects/68baf8cd87e6498d93e7ddebe48a700e => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 08:45:34.413177 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab85b77-abe9-473f-b799-dae147091eb6 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:34.413580 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab85b77-abe9-473f-b799-dae147091eb6 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:34.413779 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16349bb-f5d7-4b26-a6f2-d632c5bde5b7 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:34.414022 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab85b77-abe9-473f-b799-dae147091eb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:34.414131 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16349bb-f5d7-4b26-a6f2-d632c5bde5b7 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:34.414328 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16349bb-f5d7-4b26-a6f2-d632c5bde5b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:34.434683 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-2ab85b77-abe9-473f-b799-dae147091eb6 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:34.435783 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-e16349bb-f5d7-4b26-a6f2-d632c5bde5b7 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:34.436119 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-2ab85b77-abe9-473f-b799-dae147091eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:34.438217 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-e16349bb-f5d7-4b26-a6f2-d632c5bde5b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:34.508223 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 757/1522] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 08:45:34.516899 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 766/1523] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:45:34.517477 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff4b060-d41d-4e05-96aa-febc49d6ae88 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:34.517804 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff4b060-d41d-4e05-96aa-febc49d6ae88 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:34.518097 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff4b060-d41d-4e05-96aa-febc49d6ae88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:34.524779 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60590928-dc81-493d-86d6-3705805be1c1 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:34.525163 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60590928-dc81-493d-86d6-3705805be1c1 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:34.525643 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60590928-dc81-493d-86d6-3705805be1c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:34.545318 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-aff4b060-d41d-4e05-96aa-febc49d6ae88 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:34.546427 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-60590928-dc81-493d-86d6-3705805be1c1 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:34.546784 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-aff4b060-d41d-4e05-96aa-febc49d6ae88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:34.547765 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-60590928-dc81-493d-86d6-3705805be1c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:34.555892 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 767/1524] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:45:34.560722 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60590928-dc81-493d-86d6-3705805be1c1 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:34.560966 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 758/1525] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 08:45:34.562012 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff4b060-d41d-4e05-96aa-febc49d6ae88 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:34.566804 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:34.572533 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60590928-dc81-493d-86d6-3705805be1c1 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:34.573185 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff4b060-d41d-4e05-96aa-febc49d6ae88 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:34.578206 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ffce27-5fb7-4f72-afc8-9258fad7b8af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:34.578960 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ffce27-5fb7-4f72-afc8-9258fad7b8af admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:34.579365 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ffce27-5fb7-4f72-afc8-9258fad7b8af admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:34.579665 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6ee065-bc14-4765-82b0-d0da22893659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:34.579816 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ffce27-5fb7-4f72-afc8-9258fad7b8af admin admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:34.580575 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ffce27-5fb7-4f72-afc8-9258fad7b8af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:45:34.580954 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ffce27-5fb7-4f72-afc8-9258fad7b8af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:34.581496 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6ee065-bc14-4765-82b0-d0da22893659 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:34.581958 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6ee065-bc14-4765-82b0-d0da22893659 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:34.582466 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6ee065-bc14-4765-82b0-d0da22893659 admin admin] PATH_INFO: `/v3/domains` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:34.582888 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ffce27-5fb7-4f72-afc8-9258fad7b8af admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:34.583546 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6ee065-bc14-4765-82b0-d0da22893659 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:45:34.584202 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6ee065-bc14-4765-82b0-d0da22893659 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:34.587486 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6ee065-bc14-4765-82b0-d0da22893659 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:34.602274 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 768/1526] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 08:45:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 08:45:34.603308 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ffce27-5fb7-4f72-afc8-9258fad7b8af admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:34.614293 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 759/1527] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 08:45:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:45:34.617736 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ffce27-5fb7-4f72-afc8-9258fad7b8af admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:34.618494 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6ee065-bc14-4765-82b0-d0da22893659 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:34.623342 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e11803-0381-44f0-82c4-482b652f307c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:34.624075 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e11803-0381-44f0-82c4-482b652f307c admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:34.624365 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e11803-0381-44f0-82c4-482b652f307c admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:34.625168 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e11803-0381-44f0-82c4-482b652f307c admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:34.625289 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e11803-0381-44f0-82c4-482b652f307c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:34.627327 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e11803-0381-44f0-82c4-482b652f307c admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:34.630107 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6ee065-bc14-4765-82b0-d0da22893659 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:34.632694 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:34.642457 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599afc79-25c7-47fe-a6fc-8410cab625e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:34.643520 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599afc79-25c7-47fe-a6fc-8410cab625e1 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:34.644745 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599afc79-25c7-47fe-a6fc-8410cab625e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:34.645609 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599afc79-25c7-47fe-a6fc-8410cab625e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:34.646985 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599afc79-25c7-47fe-a6fc-8410cab625e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:34.650981 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599afc79-25c7-47fe-a6fc-8410cab625e1 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:34.661935 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 769/1528] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 08:45:34 2022] POST /identity/v3/projects => generated 383 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 08:45:34.664676 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e11803-0381-44f0-82c4-482b652f307c admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:34.693975 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 760/1529] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 08:45:34 2022] POST /identity/v3/projects => generated 365 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:45:34.698499 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599afc79-25c7-47fe-a6fc-8410cab625e1 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:34.731474 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e11803-0381-44f0-82c4-482b652f307c admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:34.780273 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599afc79-25c7-47fe-a6fc-8410cab625e1 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:34.785011 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fdfab8-1004-41d6-baae-d30c5c379804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:34.785671 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fdfab8-1004-41d6-baae-d30c5c379804 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:34.785976 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fdfab8-1004-41d6-baae-d30c5c379804 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:34.786336 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fdfab8-1004-41d6-baae-d30c5c379804 admin admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:34.787020 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fdfab8-1004-41d6-baae-d30c5c379804 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:34.789200 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fdfab8-1004-41d6-baae-d30c5c379804 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:34.808062 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddde507-d09e-4e4f-9c0e-507be248e40e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:34.809046 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddde507-d09e-4e4f-9c0e-507be248e40e admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:34.809634 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddde507-d09e-4e4f-9c0e-507be248e40e admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:34.810146 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddde507-d09e-4e4f-9c0e-507be248e40e admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:34.810896 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddde507-d09e-4e4f-9c0e-507be248e40e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:34.812707 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddde507-d09e-4e4f-9c0e-507be248e40e admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:34.822233 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 761/1530] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:45:34 2022] POST /identity/v3/users => generated 283 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 08:45:34.823606 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fdfab8-1004-41d6-baae-d30c5c379804 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:34.839672 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fdfab8-1004-41d6-baae-d30c5c379804 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:34.844443 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 770/1531] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:45:34 2022] POST /identity/v3/users => generated 292 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 08:45:34.845551 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddde507-d09e-4e4f-9c0e-507be248e40e admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:34.848543 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-fddde507-d09e-4e4f-9c0e-507be248e40e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:34.852363 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a1d62-0be8-48e9-b48f-4e55cf9e0a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:34.853146 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a1d62-0be8-48e9-b48f-4e55cf9e0a2d admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:34.853402 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a1d62-0be8-48e9-b48f-4e55cf9e0a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:34.853697 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a1d62-0be8-48e9-b48f-4e55cf9e0a2d admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:34.854406 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091a1d62-0be8-48e9-b48f-4e55cf9e0a2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:34.856108 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091a1d62-0be8-48e9-b48f-4e55cf9e0a2d admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:34.863642 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddde507-d09e-4e4f-9c0e-507be248e40e admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:34.868654 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 762/1532] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:45:34 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 08:45:34.869586 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a1d62-0be8-48e9-b48f-4e55cf9e0a2d admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:34.872248 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9a3b1c-dacd-48c0-847e-dd011a3234f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:34.872980 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a3b1c-dacd-48c0-847e-dd011a3234f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:34.873338 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a3b1c-dacd-48c0-847e-dd011a3234f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:34.873720 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a3b1c-dacd-48c0-847e-dd011a3234f4 admin admin] PATH_INFO: `/v3/users/f5dd1b8ef9524ef6a5a7f19dd7b1b2be` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:34.874970 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9a3b1c-dacd-48c0-847e-dd011a3234f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5dd1b8ef9524ef6a5a7f19dd7b1b2be)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:34.877035 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9a3b1c-dacd-48c0-847e-dd011a3234f4 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:34.880148 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a1d62-0be8-48e9-b48f-4e55cf9e0a2d admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:34.886836 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdddba58-3622-4b00-a3e9-dfb29171bf06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:34.888113 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdddba58-3622-4b00-a3e9-dfb29171bf06 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:34.888479 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdddba58-3622-4b00-a3e9-dfb29171bf06 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:34.888820 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdddba58-3622-4b00-a3e9-dfb29171bf06 admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:34.890153 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdddba58-3622-4b00-a3e9-dfb29171bf06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:34.891827 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdddba58-3622-4b00-a3e9-dfb29171bf06 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:34.904078 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 763/1533] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:45:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:45:34.904664 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdddba58-3622-4b00-a3e9-dfb29171bf06 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:34.917398 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdddba58-3622-4b00-a3e9-dfb29171bf06 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:34.927244 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1d3ae2-0e97-4234-a980-2ca574841ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:34.928053 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d3ae2-0e97-4234-a980-2ca574841ae6 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:34.928487 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d3ae2-0e97-4234-a980-2ca574841ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:34.929443 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d3ae2-0e97-4234-a980-2ca574841ae6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:34.930538 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1d3ae2-0e97-4234-a980-2ca574841ae6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:34.941709 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1d3ae2-0e97-4234-a980-2ca574841ae6 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:34.947785 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 764/1534] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:45:34 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:45:34.947785 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1d3ae2-0e97-4234-a980-2ca574841ae6 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:34.955961 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-2a9a3b1c-dacd-48c0-847e-dd011a3234f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5dd1b8ef9524ef6a5a7f19dd7b1b2be'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:34.956624 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-2a9a3b1c-dacd-48c0-847e-dd011a3234f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5dd1b8ef9524ef6a5a7f19dd7b1b2be'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:34.958450 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1d3ae2-0e97-4234-a980-2ca574841ae6 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:34.963541 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995bf7f5-347f-4788-95f1-66a6c00487a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:34.964150 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995bf7f5-347f-4788-95f1-66a6c00487a9 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:34.964407 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995bf7f5-347f-4788-95f1-66a6c00487a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:34.964678 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995bf7f5-347f-4788-95f1-66a6c00487a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:34.965320 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-2a9a3b1c-dacd-48c0-847e-dd011a3234f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5dd1b8ef9524ef6a5a7f19dd7b1b2be'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:34.965423 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995bf7f5-347f-4788-95f1-66a6c00487a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:34.967142 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995bf7f5-347f-4788-95f1-66a6c00487a9 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:34.978019 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 765/1535] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:45:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:45:34.978648 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995bf7f5-347f-4788-95f1-66a6c00487a9 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:34.980315 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-995bf7f5-347f-4788-95f1-66a6c00487a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:34.980975 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-2a9a3b1c-dacd-48c0-847e-dd011a3234f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5dd1b8ef9524ef6a5a7f19dd7b1b2be'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:34.984228 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=86,keystone:INSERT=5,keystone:DELETE=9 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:34.990891 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-2a9a3b1c-dacd-48c0-847e-dd011a3234f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5dd1b8ef9524ef6a5a7f19dd7b1b2be'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:34.994137 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995bf7f5-347f-4788-95f1-66a6c00487a9 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:34.999785 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 771/1536] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 08:45:34 2022] DELETE /identity/v3/users/f5dd1b8ef9524ef6a5a7f19dd7b1b2be => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:35.000747 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9a3b1c-dacd-48c0-847e-dd011a3234f4 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:35.002294 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d55b19-f293-4b2e-9b36-8f99bf270a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:35.002681 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-2a9a3b1c-dacd-48c0-847e-dd011a3234f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:35.003547 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d55b19-f293-4b2e-9b36-8f99bf270a0f admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:35.003842 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d55b19-f293-4b2e-9b36-8f99bf270a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:35.004130 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d55b19-f293-4b2e-9b36-8f99bf270a0f admin admin] PATH_INFO: `/v3/projects/c005fb5fbbf3481d832e40f3962b4cb5/users/530d2cfb58d84be9a14120f02955c639/roles/3248f0268738489b883041851d681bb7` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:35.020784 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d55b19-f293-4b2e-9b36-8f99bf270a0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c005fb5fbbf3481d832e40f3962b4cb5, user_id=530d2cfb58d84be9a14120f02955c639, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:35.025010 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d55b19-f293-4b2e-9b36-8f99bf270a0f admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:35.041147 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 766/1537] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:45:34 2022] PUT /identity/v3/projects/c005fb5fbbf3481d832e40f3962b4cb5/users/530d2cfb58d84be9a14120f02955c639/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:35.051802 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a42aaf-e82d-447c-8df8-dcf0be160278 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:35.052162 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a42aaf-e82d-447c-8df8-dcf0be160278 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:35.052508 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a42aaf-e82d-447c-8df8-dcf0be160278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:35.074310 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-69a42aaf-e82d-447c-8df8-dcf0be160278 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:35.075527 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-69a42aaf-e82d-447c-8df8-dcf0be160278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:35.077472 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9a3b1c-dacd-48c0-847e-dd011a3234f4 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:35.170745 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:35.175787 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ab8e54-290a-4df1-b89a-8c46031f02db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:35.176557 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab8e54-290a-4df1-b89a-8c46031f02db admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:35.176912 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab8e54-290a-4df1-b89a-8c46031f02db admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:35.177253 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab8e54-290a-4df1-b89a-8c46031f02db admin admin] PATH_INFO: `/v3/projects/c3311f1ce9094338be6b41b73ae00f2b/users/9b70c56a09a6493282e7dc6196131949/roles/3248f0268738489b883041851d681bb7` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:35.191594 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 767/1538] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:45:35.193107 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ad4ff7-6f4b-4a5a-b879-290d95d1bdab None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:35.193399 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ad4ff7-6f4b-4a5a-b879-290d95d1bdab None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:35.193763 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ad4ff7-6f4b-4a5a-b879-290d95d1bdab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:35.207878 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ab8e54-290a-4df1-b89a-8c46031f02db admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3311f1ce9094338be6b41b73ae00f2b, user_id=9b70c56a09a6493282e7dc6196131949, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:35.210692 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ab8e54-290a-4df1-b89a-8c46031f02db admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:35.235080 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 772/1539] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:45:34 2022] PUT /identity/v3/projects/c3311f1ce9094338be6b41b73ae00f2b/users/9b70c56a09a6493282e7dc6196131949/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:35.235080 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ab8e54-290a-4df1-b89a-8c46031f02db admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:35.244102 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-13ab8e54-290a-4df1-b89a-8c46031f02db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:35.255445 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-25ad4ff7-6f4b-4a5a-b879-290d95d1bdab None None] MFA Rules not processed for user `530d2cfb58d84be9a14120f02955c639`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:35.256897 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-25ad4ff7-6f4b-4a5a-b879-290d95d1bdab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:35.313569 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ab8e54-290a-4df1-b89a-8c46031f02db admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:35.317832 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438d34b7-e210-494a-b11d-e5ba6653ba4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:35.318595 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d34b7-e210-494a-b11d-e5ba6653ba4c service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:35.319050 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d34b7-e210-494a-b11d-e5ba6653ba4c service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:35.319357 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d34b7-e210-494a-b11d-e5ba6653ba4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:35.341947 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 768/1540] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:45:35.343627 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b8f5df-9362-447e-a358-6bca151eab6c None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:35.344047 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b8f5df-9362-447e-a358-6bca151eab6c None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:35.344535 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b8f5df-9362-447e-a358-6bca151eab6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:35.364316 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:35.377235 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-a5b8f5df-9362-447e-a358-6bca151eab6c None None] MFA Rules not processed for user `9b70c56a09a6493282e7dc6196131949`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:35.378706 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-a5b8f5df-9362-447e-a358-6bca151eab6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:35.390980 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438d34b7-e210-494a-b11d-e5ba6653ba4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:35.392581 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438d34b7-e210-494a-b11d-e5ba6653ba4c service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:35.405026 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 773/1541] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:45:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:45:35.405988 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438d34b7-e210-494a-b11d-e5ba6653ba4c service neutron] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:35.417283 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438d34b7-e210-494a-b11d-e5ba6653ba4c service neutron] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:35.422836 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af206fcd-855c-471d-bb17-e0edd2f5efd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:35.423493 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af206fcd-855c-471d-bb17-e0edd2f5efd9 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:35.423719 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af206fcd-855c-471d-bb17-e0edd2f5efd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:35.423924 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af206fcd-855c-471d-bb17-e0edd2f5efd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:35.424766 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af206fcd-855c-471d-bb17-e0edd2f5efd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:35.427883 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af206fcd-855c-471d-bb17-e0edd2f5efd9 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:35.455178 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 769/1542] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:45:35.459655 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b8f5df-9362-447e-a358-6bca151eab6c None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:35.480704 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 774/1543] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 08:45:35 2022] POST /identity/v3/projects => generated 365 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:45:35.484979 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af206fcd-855c-471d-bb17-e0edd2f5efd9 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:35.486041 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b8f5df-9362-447e-a358-6bca151eab6c None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:35.609724 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af206fcd-855c-471d-bb17-e0edd2f5efd9 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:35.616087 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf906d9-ba9e-42a4-91d5-d8d0a2303680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:35.617027 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf906d9-ba9e-42a4-91d5-d8d0a2303680 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:35.617970 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf906d9-ba9e-42a4-91d5-d8d0a2303680 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:35.618361 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf906d9-ba9e-42a4-91d5-d8d0a2303680 admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:35.619156 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf906d9-ba9e-42a4-91d5-d8d0a2303680 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:35.622221 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf906d9-ba9e-42a4-91d5-d8d0a2303680 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:35.623777 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0b2751-dd23-416e-8446-30cbe1d5dd3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:35.624414 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0b2751-dd23-416e-8446-30cbe1d5dd3c admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:35.624744 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0b2751-dd23-416e-8446-30cbe1d5dd3c admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:35.625022 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0b2751-dd23-416e-8446-30cbe1d5dd3c admin admin] PATH_INFO: `/v3/projects` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:35.625877 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0b2751-dd23-416e-8446-30cbe1d5dd3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:35.631774 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0b2751-dd23-416e-8446-30cbe1d5dd3c admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:35.664707 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 775/1544] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:45:35 2022] POST /identity/v3/users => generated 289 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:45:35.666229 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf906d9-ba9e-42a4-91d5-d8d0a2303680 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:35.681141 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf906d9-ba9e-42a4-91d5-d8d0a2303680 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:35.681750 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:35.690747 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a06235c-19a9-4b21-8fe2-9c89b455cdc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:35.691707 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a06235c-19a9-4b21-8fe2-9c89b455cdc1 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:35.692078 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a06235c-19a9-4b21-8fe2-9c89b455cdc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:35.692454 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a06235c-19a9-4b21-8fe2-9c89b455cdc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:35.693346 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a06235c-19a9-4b21-8fe2-9c89b455cdc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:35.696101 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a06235c-19a9-4b21-8fe2-9c89b455cdc1 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:35.711090 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 770/1545] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 08:45:35 2022] POST /identity/v3/projects => generated 383 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:45:35.712178 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0b2751-dd23-416e-8446-30cbe1d5dd3c admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:35.716770 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 776/1546] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:45:35 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 08:45:35.718267 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a06235c-19a9-4b21-8fe2-9c89b455cdc1 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:35.733388 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a06235c-19a9-4b21-8fe2-9c89b455cdc1 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:35.830866 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2105dd3d-b03b-486a-850f-adfaceaf91f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:35.831784 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2105dd3d-b03b-486a-850f-adfaceaf91f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:35.833442 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2105dd3d-b03b-486a-850f-adfaceaf91f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:35.834340 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2105dd3d-b03b-486a-850f-adfaceaf91f7 admin admin] PATH_INFO: `/v3/projects/8a84972afc3b47f7b3c5603ca539e717/users/25b0cf8063774ebf8ef11d318a12147b/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:35.853176 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0b2751-dd23-416e-8446-30cbe1d5dd3c admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:35.859699 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6f647e-94cf-4bc7-a6d1-09d159e3d226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:35.860751 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f647e-94cf-4bc7-a6d1-09d159e3d226 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:35.861233 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f647e-94cf-4bc7-a6d1-09d159e3d226 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:35.861850 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f647e-94cf-4bc7-a6d1-09d159e3d226 admin admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:35.862858 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6f647e-94cf-4bc7-a6d1-09d159e3d226 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:35.865210 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6f647e-94cf-4bc7-a6d1-09d159e3d226 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:35.868766 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2105dd3d-b03b-486a-850f-adfaceaf91f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a84972afc3b47f7b3c5603ca539e717, user_id=25b0cf8063774ebf8ef11d318a12147b, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:35.871498 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2105dd3d-b03b-486a-850f-adfaceaf91f7 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:35.904643 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 777/1547] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:45:35 2022] PUT /identity/v3/projects/8a84972afc3b47f7b3c5603ca539e717/users/25b0cf8063774ebf8ef11d318a12147b/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:35.909743 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 771/1548] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:45:35 2022] POST /identity/v3/users => generated 298 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:45:35.913346 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6f647e-94cf-4bc7-a6d1-09d159e3d226 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:35.913471 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2105dd3d-b03b-486a-850f-adfaceaf91f7 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:35.926530 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6f647e-94cf-4bc7-a6d1-09d159e3d226 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:35.958618 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:36.098964 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2105dd3d-b03b-486a-850f-adfaceaf91f7 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:36.105492 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a85572-1709-4b7d-8014-49759001d374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:36.106144 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a85572-1709-4b7d-8014-49759001d374 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:36.106424 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a85572-1709-4b7d-8014-49759001d374 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:36.106697 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a85572-1709-4b7d-8014-49759001d374 admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:36.107340 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a85572-1709-4b7d-8014-49759001d374 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:36.110421 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a85572-1709-4b7d-8014-49759001d374 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:36.114307 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bb22f9-3b78-4622-8af7-6220c12cd5ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:36.115335 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bb22f9-3b78-4622-8af7-6220c12cd5ea admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:36.115624 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bb22f9-3b78-4622-8af7-6220c12cd5ea admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:36.115900 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bb22f9-3b78-4622-8af7-6220c12cd5ea admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:36.116705 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bb22f9-3b78-4622-8af7-6220c12cd5ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:36.121157 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bb22f9-3b78-4622-8af7-6220c12cd5ea admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:36.139329 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 772/1549] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:45:35 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 08:45:36.139329 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a85572-1709-4b7d-8014-49759001d374 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:36.148939 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-99a85572-1709-4b7d-8014-49759001d374 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:36.154013 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 778/1550] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:45:35 2022] GET /identity/v3/roles => generated 1420 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:45:36.154013 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bb22f9-3b78-4622-8af7-6220c12cd5ea admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:36.186099 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a85572-1709-4b7d-8014-49759001d374 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:36.186205 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bb22f9-3b78-4622-8af7-6220c12cd5ea admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:36.202700 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c444730-5b59-4210-a6c3-0ffe0230dac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:36.206073 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c444730-5b59-4210-a6c3-0ffe0230dac4 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:36.206361 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c444730-5b59-4210-a6c3-0ffe0230dac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:36.206644 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c444730-5b59-4210-a6c3-0ffe0230dac4 admin admin] PATH_INFO: `/v3/projects/0602f0f391444439a1a1d36a628ed1c1/users/fa7597f6a47c4749b86528ef2a1faf48/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:36.207647 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf23cce-d4c7-43a8-949d-63ff5f851618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:36.208303 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf23cce-d4c7-43a8-949d-63ff5f851618 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:36.208579 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf23cce-d4c7-43a8-949d-63ff5f851618 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:36.208850 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf23cce-d4c7-43a8-949d-63ff5f851618 admin admin] PATH_INFO: `/v3/projects/d30615a9e44047daa7af8e9bc897681d` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:36.212011 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf23cce-d4c7-43a8-949d-63ff5f851618 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d30615a9e44047daa7af8e9bc897681d)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:36.219071 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf23cce-d4c7-43a8-949d-63ff5f851618 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:36.247375 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c444730-5b59-4210-a6c3-0ffe0230dac4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0602f0f391444439a1a1d36a628ed1c1, user_id=fa7597f6a47c4749b86528ef2a1faf48, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:36.250778 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c444730-5b59-4210-a6c3-0ffe0230dac4 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:36.272468 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 779/1551] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:45:36 2022] PUT /identity/v3/projects/0602f0f391444439a1a1d36a628ed1c1/users/fa7597f6a47c4749b86528ef2a1faf48/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:36.273969 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c444730-5b59-4210-a6c3-0ffe0230dac4 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:36.293446 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-7cf23cce-d4c7-43a8-949d-63ff5f851618 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd30615a9e44047daa7af8e9bc897681d', 'request_id': 'req-7cf23cce-d4c7-43a8-949d-63ff5f851618'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:36.300906 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-7cf23cce-d4c7-43a8-949d-63ff5f851618 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd30615a9e44047daa7af8e9bc897681d', 'request_id': 'req-7cf23cce-d4c7-43a8-949d-63ff5f851618'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:36.310470 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-7cf23cce-d4c7-43a8-949d-63ff5f851618 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd30615a9e44047daa7af8e9bc897681d', 'request_id': 'req-7cf23cce-d4c7-43a8-949d-63ff5f851618'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:36.327253 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-7cf23cce-d4c7-43a8-949d-63ff5f851618 admin admin] The token cache is being invalidate because project d30615a9e44047daa7af8e9bc897681d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 08:45:36.327681 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-7cf23cce-d4c7-43a8-949d-63ff5f851618 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:36.331332 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-7cf23cce-d4c7-43a8-949d-63ff5f851618 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:36.334543 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 773/1552] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 08:45:36 2022] DELETE /identity/v3/projects/d30615a9e44047daa7af8e9bc897681d => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:36.335761 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf23cce-d4c7-43a8-949d-63ff5f851618 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:36.337549 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-7cf23cce-d4c7-43a8-949d-63ff5f851618 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:36.342611 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:INSERT=5,keystone:SELECT=88,keystone:DELETE=7 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:36.354021 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c444730-5b59-4210-a6c3-0ffe0230dac4 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:36.417769 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf23cce-d4c7-43a8-949d-63ff5f851618 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:36.423202 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6036df-8c37-445c-a75f-22e6dd07af16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:36.423899 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6036df-8c37-445c-a75f-22e6dd07af16 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:36.424276 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6036df-8c37-445c-a75f-22e6dd07af16 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:36.424534 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6036df-8c37-445c-a75f-22e6dd07af16 admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:36.425263 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6036df-8c37-445c-a75f-22e6dd07af16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:36.427297 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6036df-8c37-445c-a75f-22e6dd07af16 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:36.427535 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de16dbf1-98d1-42c7-87b6-fd8c76800f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:36.428202 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de16dbf1-98d1-42c7-87b6-fd8c76800f4b admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:36.428614 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de16dbf1-98d1-42c7-87b6-fd8c76800f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:36.429435 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de16dbf1-98d1-42c7-87b6-fd8c76800f4b admin admin] PATH_INFO: `/v3/projects/8a84972afc3b47f7b3c5603ca539e717/users/25b0cf8063774ebf8ef11d318a12147b/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:36.431850 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de16dbf1-98d1-42c7-87b6-fd8c76800f4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a84972afc3b47f7b3c5603ca539e717, user_id=25b0cf8063774ebf8ef11d318a12147b, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:36.434405 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de16dbf1-98d1-42c7-87b6-fd8c76800f4b admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:36.440122 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 774/1553] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:45:36 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:45:36.441002 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6036df-8c37-445c-a75f-22e6dd07af16 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:36.442707 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-da6036df-8c37-445c-a75f-22e6dd07af16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:36.447142 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 780/1554] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:45:36 2022] PUT /identity/v3/projects/8a84972afc3b47f7b3c5603ca539e717/users/25b0cf8063774ebf8ef11d318a12147b/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:36.448108 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de16dbf1-98d1-42c7-87b6-fd8c76800f4b admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:36.457166 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6036df-8c37-445c-a75f-22e6dd07af16 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:36.491168 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:36.521338 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b252153a-9277-4388-b9bf-e35ca1e25438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:36.522006 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b252153a-9277-4388-b9bf-e35ca1e25438 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:36.522285 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b252153a-9277-4388-b9bf-e35ca1e25438 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:36.522588 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b252153a-9277-4388-b9bf-e35ca1e25438 admin admin] PATH_INFO: `/v3/users/f5d5dbc016b94551be445e7f5e40d480` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:36.523787 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b252153a-9277-4388-b9bf-e35ca1e25438 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5d5dbc016b94551be445e7f5e40d480)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:36.526567 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b252153a-9277-4388-b9bf-e35ca1e25438 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:36.534159 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de16dbf1-98d1-42c7-87b6-fd8c76800f4b admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:36.540854 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e04f1d2-ef5b-42de-9be7-34afcb1065c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:36.541526 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e04f1d2-ef5b-42de-9be7-34afcb1065c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:36.541838 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e04f1d2-ef5b-42de-9be7-34afcb1065c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:36.542110 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e04f1d2-ef5b-42de-9be7-34afcb1065c9 admin admin] PATH_INFO: `/v3/projects/0602f0f391444439a1a1d36a628ed1c1/users/fa7597f6a47c4749b86528ef2a1faf48/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:36.544401 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e04f1d2-ef5b-42de-9be7-34afcb1065c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0602f0f391444439a1a1d36a628ed1c1, user_id=fa7597f6a47c4749b86528ef2a1faf48, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:36.547244 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e04f1d2-ef5b-42de-9be7-34afcb1065c9 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:36.559958 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 781/1555] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:45:36 2022] PUT /identity/v3/projects/0602f0f391444439a1a1d36a628ed1c1/users/fa7597f6a47c4749b86528ef2a1faf48/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:36.560647 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f05bd9-c4f0-47f2-b3c3-3fed4858ae58 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:36.560949 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f05bd9-c4f0-47f2-b3c3-3fed4858ae58 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:36.561341 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f05bd9-c4f0-47f2-b3c3-3fed4858ae58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:36.591308 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-47f05bd9-c4f0-47f2-b3c3-3fed4858ae58 None None] MFA Rules not processed for user `25b0cf8063774ebf8ef11d318a12147b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:36.592527 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-47f05bd9-c4f0-47f2-b3c3-3fed4858ae58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:36.594644 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-b252153a-9277-4388-b9bf-e35ca1e25438 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5d5dbc016b94551be445e7f5e40d480'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:36.595294 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-b252153a-9277-4388-b9bf-e35ca1e25438 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5d5dbc016b94551be445e7f5e40d480'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:36.613498 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-b252153a-9277-4388-b9bf-e35ca1e25438 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5d5dbc016b94551be445e7f5e40d480'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:36.622374 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-b252153a-9277-4388-b9bf-e35ca1e25438 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5d5dbc016b94551be445e7f5e40d480'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:36.623092 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-b252153a-9277-4388-b9bf-e35ca1e25438 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5d5dbc016b94551be445e7f5e40d480'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:36.640578 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 775/1556] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 08:45:36 2022] DELETE /identity/v3/users/f5d5dbc016b94551be445e7f5e40d480 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:36.642141 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bfcf33-b385-4eac-b585-2ebb67cb5d43 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:36.642497 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bfcf33-b385-4eac-b585-2ebb67cb5d43 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:36.642835 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bfcf33-b385-4eac-b585-2ebb67cb5d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:36.678217 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-03bfcf33-b385-4eac-b585-2ebb67cb5d43 None None] MFA Rules not processed for user `fa7597f6a47c4749b86528ef2a1faf48`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:36.680164 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-03bfcf33-b385-4eac-b585-2ebb67cb5d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:36.687473 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:36.697311 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 782/1557] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:36 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:45:36.710262 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f05bd9-c4f0-47f2-b3c3-3fed4858ae58 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:36.714603 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-47f05bd9-c4f0-47f2-b3c3-3fed4858ae58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:36.776113 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 776/1558] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:36 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:45:36.787074 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bfcf33-b385-4eac-b585-2ebb67cb5d43 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:36.789245 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-03bfcf33-b385-4eac-b585-2ebb67cb5d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:36.843733 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f05bd9-c4f0-47f2-b3c3-3fed4858ae58 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:36.922758 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bfcf33-b385-4eac-b585-2ebb67cb5d43 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:36.932643 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccde260c-285c-4d51-8d8d-6c006625eae0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:36.933295 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde260c-285c-4d51-8d8d-6c006625eae0 service cinder] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:36.941417 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde260c-285c-4d51-8d8d-6c006625eae0 service cinder] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:36.941830 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde260c-285c-4d51-8d8d-6c006625eae0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:36.966590 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49bd457-40db-49bc-9b7e-a3f0eccc3101 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:36.967518 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49bd457-40db-49bc-9b7e-a3f0eccc3101 service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:36.969077 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49bd457-40db-49bc-9b7e-a3f0eccc3101 service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:36.969649 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49bd457-40db-49bc-9b7e-a3f0eccc3101 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:36.974676 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccde260c-285c-4d51-8d8d-6c006625eae0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:36.977324 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccde260c-285c-4d51-8d8d-6c006625eae0 service cinder] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:37.011975 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 777/1559] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:45:36 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 08:45:37.026807 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:37.168159 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49bd457-40db-49bc-9b7e-a3f0eccc3101 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:37.170105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49bd457-40db-49bc-9b7e-a3f0eccc3101 service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:37.184057 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 783/1560] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:45:36 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 08:45:37.248174 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccde260c-285c-4d51-8d8d-6c006625eae0 service cinder] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:37.250362 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-ccde260c-285c-4d51-8d8d-6c006625eae0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:37.426241 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49bd457-40db-49bc-9b7e-a3f0eccc3101 service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:37.437302 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49bd457-40db-49bc-9b7e-a3f0eccc3101 service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:37.447657 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406161cd-48ce-4261-a1c7-d7f8f9398df3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:37.451718 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406161cd-48ce-4261-a1c7-d7f8f9398df3 service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:37.452136 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406161cd-48ce-4261-a1c7-d7f8f9398df3 service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:37.452683 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406161cd-48ce-4261-a1c7-d7f8f9398df3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:37.454704 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccde260c-285c-4d51-8d8d-6c006625eae0 service cinder] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:37.455003 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-406161cd-48ce-4261-a1c7-d7f8f9398df3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:37.465429 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc86fb7-8426-432e-9dab-3e3e465859c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:37.466405 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc86fb7-8426-432e-9dab-3e3e465859c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:37.466702 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc86fb7-8426-432e-9dab-3e3e465859c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:37.466975 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc86fb7-8426-432e-9dab-3e3e465859c8 admin admin] PATH_INFO: `/v3/projects/8a6a6477e96640d29aa86f602ffd3273` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:37.470769 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc86fb7-8426-432e-9dab-3e3e465859c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a6a6477e96640d29aa86f602ffd3273)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:37.474831 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc86fb7-8426-432e-9dab-3e3e465859c8 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:37.563419 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:37.566332 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-fdc86fb7-8426-432e-9dab-3e3e465859c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a6a6477e96640d29aa86f602ffd3273', 'request_id': 'req-fdc86fb7-8426-432e-9dab-3e3e465859c8'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:37.579924 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-fdc86fb7-8426-432e-9dab-3e3e465859c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a6a6477e96640d29aa86f602ffd3273', 'request_id': 'req-fdc86fb7-8426-432e-9dab-3e3e465859c8'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:37.593406 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-fdc86fb7-8426-432e-9dab-3e3e465859c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a6a6477e96640d29aa86f602ffd3273', 'request_id': 'req-fdc86fb7-8426-432e-9dab-3e3e465859c8'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:37.600523 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406161cd-48ce-4261-a1c7-d7f8f9398df3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:37.602383 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406161cd-48ce-4261-a1c7-d7f8f9398df3 service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:37.613174 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 784/1561] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:45:37 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 08:45:37.614014 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406161cd-48ce-4261-a1c7-d7f8f9398df3 service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:37.616708 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-fdc86fb7-8426-432e-9dab-3e3e465859c8 admin admin] The token cache is being invalidate because project 8a6a6477e96640d29aa86f602ffd3273 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 08:45:37.617120 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-fdc86fb7-8426-432e-9dab-3e3e465859c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:37.617932 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-fdc86fb7-8426-432e-9dab-3e3e465859c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:37.620304 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 778/1562] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 08:45:37 2022] DELETE /identity/v3/projects/8a6a6477e96640d29aa86f602ffd3273 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:37.624603 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b7fe8b-48df-4cca-8189-c40f714c31ef None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:37.624994 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b7fe8b-48df-4cca-8189-c40f714c31ef None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:37.625376 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b7fe8b-48df-4cca-8189-c40f714c31ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:37.627744 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406161cd-48ce-4261-a1c7-d7f8f9398df3 service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:37.672650 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-01b7fe8b-48df-4cca-8189-c40f714c31ef None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:37.674005 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-01b7fe8b-48df-4cca-8189-c40f714c31ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:37.723417 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4776e79e-a404-42d3-8d0c-e4aa3f06b127 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:37.724753 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4776e79e-a404-42d3-8d0c-e4aa3f06b127 service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:37.725839 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4776e79e-a404-42d3-8d0c-e4aa3f06b127 service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:37.726381 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4776e79e-a404-42d3-8d0c-e4aa3f06b127 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:37.730291 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-4776e79e-a404-42d3-8d0c-e4aa3f06b127 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:37.821434 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 779/1563] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:45:37.829188 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f6771e-4e51-4240-a847-68964daaab47 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:37.829484 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f6771e-4e51-4240-a847-68964daaab47 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:37.829797 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f6771e-4e51-4240-a847-68964daaab47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:37.854713 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4776e79e-a404-42d3-8d0c-e4aa3f06b127 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:37.858146 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4776e79e-a404-42d3-8d0c-e4aa3f06b127 service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:37.873090 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 785/1564] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:45:37 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:45:37.895277 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-b8f6771e-4e51-4240-a847-68964daaab47 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:37.897022 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-b8f6771e-4e51-4240-a847-68964daaab47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:37.920693 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 780/1565] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:45:37.922780 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4776e79e-a404-42d3-8d0c-e4aa3f06b127 service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:37.930825 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:37.943343 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4776e79e-a404-42d3-8d0c-e4aa3f06b127 service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:37.960998 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22942b37-4433-4086-96b8-ba15e25877ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:37.963401 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22942b37-4433-4086-96b8-ba15e25877ed admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:37.965195 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22942b37-4433-4086-96b8-ba15e25877ed admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:37.965587 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22942b37-4433-4086-96b8-ba15e25877ed admin admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:37.967814 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22942b37-4433-4086-96b8-ba15e25877ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:45:37.969109 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22942b37-4433-4086-96b8-ba15e25877ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:37.972825 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22942b37-4433-4086-96b8-ba15e25877ed admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:38.010035 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f6771e-4e51-4240-a847-68964daaab47 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:38.012111 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 786/1566] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 08:45:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:45:38.026974 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f6771e-4e51-4240-a847-68964daaab47 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:38.033799 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1def09e-5baf-4a17-a3d8-8d6145d90a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:38.034762 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1def09e-5baf-4a17-a3d8-8d6145d90a71 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:38.035052 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1def09e-5baf-4a17-a3d8-8d6145d90a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:38.035333 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1def09e-5baf-4a17-a3d8-8d6145d90a71 admin admin] PATH_INFO: `/v3/projects` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:38.035983 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1def09e-5baf-4a17-a3d8-8d6145d90a71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:38.039553 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1def09e-5baf-4a17-a3d8-8d6145d90a71 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:38.080464 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 781/1567] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 08:45:38 2022] POST /identity/v3/projects => generated 383 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 08:45:38.082486 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22942b37-4433-4086-96b8-ba15e25877ed admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:38.097559 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22942b37-4433-4086-96b8-ba15e25877ed admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:38.200926 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719bd9fc-a6e2-4e87-a532-3182d59415ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:38.201856 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719bd9fc-a6e2-4e87-a532-3182d59415ca admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:38.202299 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719bd9fc-a6e2-4e87-a532-3182d59415ca admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:38.202977 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719bd9fc-a6e2-4e87-a532-3182d59415ca admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:38.204248 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719bd9fc-a6e2-4e87-a532-3182d59415ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:38.206714 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719bd9fc-a6e2-4e87-a532-3182d59415ca admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:38.247365 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 787/1568] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:45:38 2022] POST /identity/v3/users => generated 292 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:45:38.251414 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1def09e-5baf-4a17-a3d8-8d6145d90a71 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:38.264379 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1def09e-5baf-4a17-a3d8-8d6145d90a71 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:38.276683 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f618501f-157f-48b4-ab54-193f01eeef7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:38.277338 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f618501f-157f-48b4-ab54-193f01eeef7b admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:38.277616 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f618501f-157f-48b4-ab54-193f01eeef7b admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:38.277916 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f618501f-157f-48b4-ab54-193f01eeef7b admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:38.278562 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f618501f-157f-48b4-ab54-193f01eeef7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:38.280624 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f618501f-157f-48b4-ab54-193f01eeef7b admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:38.295832 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 782/1569] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:45:38 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:45:38.297794 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719bd9fc-a6e2-4e87-a532-3182d59415ca admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:38.310043 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719bd9fc-a6e2-4e87-a532-3182d59415ca admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:38.315592 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34a2d10-5f33-4bbe-ba59-1da4dd8fa20b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:38.316379 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a2d10-5f33-4bbe-ba59-1da4dd8fa20b admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:38.316804 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a2d10-5f33-4bbe-ba59-1da4dd8fa20b admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:38.317259 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a2d10-5f33-4bbe-ba59-1da4dd8fa20b admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:38.318138 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34a2d10-5f33-4bbe-ba59-1da4dd8fa20b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:38.320725 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34a2d10-5f33-4bbe-ba59-1da4dd8fa20b admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:38.336146 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 788/1570] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:45:38 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:45:38.340407 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f618501f-157f-48b4-ab54-193f01eeef7b admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:38.366380 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f618501f-157f-48b4-ab54-193f01eeef7b admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:38.374395 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c62ac7b-c591-4c75-ace4-006fc244ad14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:38.375245 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62ac7b-c591-4c75-ace4-006fc244ad14 admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:38.375598 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62ac7b-c591-4c75-ace4-006fc244ad14 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:38.375947 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62ac7b-c591-4c75-ace4-006fc244ad14 admin admin] PATH_INFO: `/v3/projects/00194da62b0b4c6a8b512557c043f0ca/users/d596c804b0ca4bb6b2ddff51afb9af6d/roles/3248f0268738489b883041851d681bb7` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:38.407757 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c62ac7b-c591-4c75-ace4-006fc244ad14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00194da62b0b4c6a8b512557c043f0ca, user_id=d596c804b0ca4bb6b2ddff51afb9af6d, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:38.410734 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c62ac7b-c591-4c75-ace4-006fc244ad14 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:38.458178 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 783/1571] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:45:38 2022] PUT /identity/v3/projects/00194da62b0b4c6a8b512557c043f0ca/users/d596c804b0ca4bb6b2ddff51afb9af6d/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 08:45:38.469020 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cbe151-a838-451b-a9aa-96eac6bbd702 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:38.469528 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cbe151-a838-451b-a9aa-96eac6bbd702 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:38.469528 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cbe151-a838-451b-a9aa-96eac6bbd702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:38.507844 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-30cbe151-a838-451b-a9aa-96eac6bbd702 None None] MFA Rules not processed for user `d596c804b0ca4bb6b2ddff51afb9af6d`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:38.509250 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-30cbe151-a838-451b-a9aa-96eac6bbd702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:38.641856 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 789/1572] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:45:38.645824 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c62ac7b-c591-4c75-ace4-006fc244ad14 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:38.754088 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c62ac7b-c591-4c75-ace4-006fc244ad14 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:38.760361 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d388a1-d369-492f-97d1-151a90133901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:38.761147 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d388a1-d369-492f-97d1-151a90133901 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:38.761419 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d388a1-d369-492f-97d1-151a90133901 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:38.761823 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d388a1-d369-492f-97d1-151a90133901 admin admin] PATH_INFO: `/v3/projects` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:38.762557 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d388a1-d369-492f-97d1-151a90133901 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:38.764563 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d388a1-d369-492f-97d1-151a90133901 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:38.789577 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:38.805416 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cbe151-a838-451b-a9aa-96eac6bbd702 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:38.805551 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 784/1573] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 08:45:38 2022] POST /identity/v3/projects => generated 387 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:45:38.816584 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cbe151-a838-451b-a9aa-96eac6bbd702 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:38.883962 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79f7cfb-415c-4411-bd4e-380a30c4033b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:38.883962 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f7cfb-415c-4411-bd4e-380a30c4033b admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:38.884395 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f7cfb-415c-4411-bd4e-380a30c4033b admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:38.884520 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f7cfb-415c-4411-bd4e-380a30c4033b admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:38.885125 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79f7cfb-415c-4411-bd4e-380a30c4033b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:38.887232 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79f7cfb-415c-4411-bd4e-380a30c4033b admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:38.897575 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:38.917843 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 790/1574] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:45:38 2022] POST /identity/v3/users => generated 300 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:45:38.919560 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d388a1-d369-492f-97d1-151a90133901 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:38.931896 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d388a1-d369-492f-97d1-151a90133901 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:38.936566 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24f796b-d514-4c0d-a170-8441dc590319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:38.937178 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f796b-d514-4c0d-a170-8441dc590319 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:38.937463 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f796b-d514-4c0d-a170-8441dc590319 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:38.937857 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f796b-d514-4c0d-a170-8441dc590319 admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:38.938496 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24f796b-d514-4c0d-a170-8441dc590319 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:38.940233 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24f796b-d514-4c0d-a170-8441dc590319 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:38.954715 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 785/1575] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:45:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:45:38.955665 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79f7cfb-415c-4411-bd4e-380a30c4033b admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:38.974625 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79f7cfb-415c-4411-bd4e-380a30c4033b admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:38.981185 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c704089-c992-4feb-9a9e-ebece81c8ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:38.982132 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c704089-c992-4feb-9a9e-ebece81c8ef7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:38.982628 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c704089-c992-4feb-9a9e-ebece81c8ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:38.982915 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c704089-c992-4feb-9a9e-ebece81c8ef7 admin admin] PATH_INFO: `/v3/projects/f0ef8aa91db840a29bf730454ee30a06/users/0d7f61efdca14285b28904261f735a35/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:38.999670 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c704089-c992-4feb-9a9e-ebece81c8ef7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0ef8aa91db840a29bf730454ee30a06, user_id=0d7f61efdca14285b28904261f735a35, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:39.003142 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c704089-c992-4feb-9a9e-ebece81c8ef7 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:39.025259 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24f796b-d514-4c0d-a170-8441dc590319 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:39.026424 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 791/1576] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:45:38 2022] PUT /identity/v3/projects/f0ef8aa91db840a29bf730454ee30a06/users/0d7f61efdca14285b28904261f735a35/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:39.038455 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24f796b-d514-4c0d-a170-8441dc590319 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:39.119375 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce10213-8c0a-45cb-a83a-ea19e63f3a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:39.120310 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce10213-8c0a-45cb-a83a-ea19e63f3a68 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:39.120867 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce10213-8c0a-45cb-a83a-ea19e63f3a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:39.121128 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce10213-8c0a-45cb-a83a-ea19e63f3a68 admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:39.122036 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce10213-8c0a-45cb-a83a-ea19e63f3a68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:39.124384 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce10213-8c0a-45cb-a83a-ea19e63f3a68 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:39.136677 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 786/1577] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:45:39 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:45:39.141248 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c704089-c992-4feb-9a9e-ebece81c8ef7 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:39.163305 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c704089-c992-4feb-9a9e-ebece81c8ef7 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:39.170965 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8993ceed-f093-4cd1-9323-1dbdd0a12cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:39.171651 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8993ceed-f093-4cd1-9323-1dbdd0a12cb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:39.171935 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8993ceed-f093-4cd1-9323-1dbdd0a12cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:39.172216 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8993ceed-f093-4cd1-9323-1dbdd0a12cb5 admin admin] PATH_INFO: `/v3/projects/f0ef8aa91db840a29bf730454ee30a06/users/0d7f61efdca14285b28904261f735a35/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:39.174221 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8993ceed-f093-4cd1-9323-1dbdd0a12cb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0ef8aa91db840a29bf730454ee30a06, user_id=0d7f61efdca14285b28904261f735a35, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:39.176928 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8993ceed-f093-4cd1-9323-1dbdd0a12cb5 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:39.190280 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 792/1578] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:45:39 2022] PUT /identity/v3/projects/f0ef8aa91db840a29bf730454ee30a06/users/0d7f61efdca14285b28904261f735a35/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:39.203034 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf21267c-f8af-484d-a037-143aa38345f2 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:39.203034 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf21267c-f8af-484d-a037-143aa38345f2 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:39.203208 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf21267c-f8af-484d-a037-143aa38345f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:39.242288 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-bf21267c-f8af-484d-a037-143aa38345f2 None None] MFA Rules not processed for user `0d7f61efdca14285b28904261f735a35`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:39.243596 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-bf21267c-f8af-484d-a037-143aa38345f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:39.326155 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 787/1579] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:39 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:45:39.338120 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8993ceed-f093-4cd1-9323-1dbdd0a12cb5 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:39.339913 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-8993ceed-f093-4cd1-9323-1dbdd0a12cb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:39.414942 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8993ceed-f093-4cd1-9323-1dbdd0a12cb5 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:39.418756 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa087db-d103-49d3-b3c4-072529999206 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:39.419385 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa087db-d103-49d3-b3c4-072529999206 service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:39.419665 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa087db-d103-49d3-b3c4-072529999206 service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:39.419939 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa087db-d103-49d3-b3c4-072529999206 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:39.431337 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa087db-d103-49d3-b3c4-072529999206 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:39.433442 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa087db-d103-49d3-b3c4-072529999206 service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:39.446529 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 793/1580] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:45:39 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:45:39.478076 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf21267c-f8af-484d-a037-143aa38345f2 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:39.621001 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf21267c-f8af-484d-a037-143aa38345f2 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:39.625750 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44c2474-2116-415e-b870-e219e2b85e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:39.625750 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44c2474-2116-415e-b870-e219e2b85e22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:39.625750 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44c2474-2116-415e-b870-e219e2b85e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:39.625750 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44c2474-2116-415e-b870-e219e2b85e22 admin admin] PATH_INFO: `/v3/users/d596c804b0ca4bb6b2ddff51afb9af6d` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:39.627307 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44c2474-2116-415e-b870-e219e2b85e22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d596c804b0ca4bb6b2ddff51afb9af6d)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:39.629101 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44c2474-2116-415e-b870-e219e2b85e22 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:39.645656 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:39.708598 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-f44c2474-2116-415e-b870-e219e2b85e22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd596c804b0ca4bb6b2ddff51afb9af6d'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:39.710785 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-f44c2474-2116-415e-b870-e219e2b85e22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd596c804b0ca4bb6b2ddff51afb9af6d'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:39.732894 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-f44c2474-2116-415e-b870-e219e2b85e22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd596c804b0ca4bb6b2ddff51afb9af6d'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:39.746082 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-f44c2474-2116-415e-b870-e219e2b85e22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd596c804b0ca4bb6b2ddff51afb9af6d'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:39.747562 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-f44c2474-2116-415e-b870-e219e2b85e22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd596c804b0ca4bb6b2ddff51afb9af6d'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:39.773950 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 788/1581] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 08:45:39 2022] DELETE /identity/v3/users/d596c804b0ca4bb6b2ddff51afb9af6d => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:39.778640 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72eabe6-8a7a-4321-910b-11d47c8735f5 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:39.779363 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72eabe6-8a7a-4321-910b-11d47c8735f5 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:39.779363 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72eabe6-8a7a-4321-910b-11d47c8735f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:39.804703 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-c72eabe6-8a7a-4321-910b-11d47c8735f5 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:39.808640 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-c72eabe6-8a7a-4321-910b-11d47c8735f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:39.878136 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 794/1582] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 08:45:39.890626 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44c2474-2116-415e-b870-e219e2b85e22 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:39.893437 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-f44c2474-2116-415e-b870-e219e2b85e22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:39.956362 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44c2474-2116-415e-b870-e219e2b85e22 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:39.960545 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84e2b8f-f6b8-410d-92fa-ea3817b2b4fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:39.961160 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84e2b8f-f6b8-410d-92fa-ea3817b2b4fb service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:39.961446 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84e2b8f-f6b8-410d-92fa-ea3817b2b4fb service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:39.961749 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84e2b8f-f6b8-410d-92fa-ea3817b2b4fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:39.970993 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84e2b8f-f6b8-410d-92fa-ea3817b2b4fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:39.972510 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84e2b8f-f6b8-410d-92fa-ea3817b2b4fb service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:39.983085 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 789/1583] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:45:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 08:45:40.255052 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72eabe6-8a7a-4321-910b-11d47c8735f5 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:40.256497 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-c72eabe6-8a7a-4321-910b-11d47c8735f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:40.279474 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72eabe6-8a7a-4321-910b-11d47c8735f5 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:40.284483 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a1e722-b27a-443c-b8e6-8202eb9910eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:40.285377 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a1e722-b27a-443c-b8e6-8202eb9910eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:40.285377 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a1e722-b27a-443c-b8e6-8202eb9910eb admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:40.285647 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a1e722-b27a-443c-b8e6-8202eb9910eb admin admin] PATH_INFO: `/v3/projects/00194da62b0b4c6a8b512557c043f0ca` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:40.287375 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a1e722-b27a-443c-b8e6-8202eb9910eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=00194da62b0b4c6a8b512557c043f0ca)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:40.290724 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a1e722-b27a-443c-b8e6-8202eb9910eb admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:40.342815 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-30a1e722-b27a-443c-b8e6-8202eb9910eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00194da62b0b4c6a8b512557c043f0ca', 'request_id': 'req-30a1e722-b27a-443c-b8e6-8202eb9910eb'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:40.343922 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:INSERT=5,keystone:SELECT=91,keystone:DELETE=4 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:40.350699 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-30a1e722-b27a-443c-b8e6-8202eb9910eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00194da62b0b4c6a8b512557c043f0ca', 'request_id': 'req-30a1e722-b27a-443c-b8e6-8202eb9910eb'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:40.358604 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-30a1e722-b27a-443c-b8e6-8202eb9910eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00194da62b0b4c6a8b512557c043f0ca', 'request_id': 'req-30a1e722-b27a-443c-b8e6-8202eb9910eb'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:40.369184 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-30a1e722-b27a-443c-b8e6-8202eb9910eb admin admin] The token cache is being invalidate because project 00194da62b0b4c6a8b512557c043f0ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 08:45:40.369418 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-30a1e722-b27a-443c-b8e6-8202eb9910eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:40.370085 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-30a1e722-b27a-443c-b8e6-8202eb9910eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:40.371574 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 795/1584] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 08:45:40 2022] DELETE /identity/v3/projects/00194da62b0b4c6a8b512557c043f0ca => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:40.375754 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84e2b8f-f6b8-410d-92fa-ea3817b2b4fb service neutron] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:40.386120 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84e2b8f-f6b8-410d-92fa-ea3817b2b4fb service neutron] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:40.441533 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5c41c1-9765-4696-b864-42fa6c1be7f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:40.442199 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5c41c1-9765-4696-b864-42fa6c1be7f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:40.442481 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5c41c1-9765-4696-b864-42fa6c1be7f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:40.442758 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5c41c1-9765-4696-b864-42fa6c1be7f4 admin admin] PATH_INFO: `/v3/users/0d7f61efdca14285b28904261f735a35` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:40.443863 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5c41c1-9765-4696-b864-42fa6c1be7f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d7f61efdca14285b28904261f735a35)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:40.446345 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5c41c1-9765-4696-b864-42fa6c1be7f4 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:40.480541 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:40.493971 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-9c5c41c1-9765-4696-b864-42fa6c1be7f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d7f61efdca14285b28904261f735a35'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:40.494658 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-9c5c41c1-9765-4696-b864-42fa6c1be7f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d7f61efdca14285b28904261f735a35'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:40.506233 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-9c5c41c1-9765-4696-b864-42fa6c1be7f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d7f61efdca14285b28904261f735a35'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:40.513977 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-9c5c41c1-9765-4696-b864-42fa6c1be7f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d7f61efdca14285b28904261f735a35'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:40.514648 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-9c5c41c1-9765-4696-b864-42fa6c1be7f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d7f61efdca14285b28904261f735a35'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:40.527617 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 790/1585] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 08:45:40 2022] DELETE /identity/v3/users/0d7f61efdca14285b28904261f735a35 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 08:45:40.731312 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a1e722-b27a-443c-b8e6-8202eb9910eb admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:40.733039 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-30a1e722-b27a-443c-b8e6-8202eb9910eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:40.754441 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a1e722-b27a-443c-b8e6-8202eb9910eb admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:40.841599 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e23a5b-fbf9-4289-8c64-edd1760187b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:40.842501 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e23a5b-fbf9-4289-8c64-edd1760187b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:40.842863 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e23a5b-fbf9-4289-8c64-edd1760187b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:40.843230 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e23a5b-fbf9-4289-8c64-edd1760187b2 admin admin] PATH_INFO: `/v3/users/530d2cfb58d84be9a14120f02955c639` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:40.844465 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e23a5b-fbf9-4289-8c64-edd1760187b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=530d2cfb58d84be9a14120f02955c639)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:40.852616 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5c41c1-9765-4696-b864-42fa6c1be7f4 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:40.854163 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-9c5c41c1-9765-4696-b864-42fa6c1be7f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:40.860542 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e23a5b-fbf9-4289-8c64-edd1760187b2 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:40.877212 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5c41c1-9765-4696-b864-42fa6c1be7f4 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:40.882872 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96102219-904a-4f13-8472-e5ce851b983e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:40.883602 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96102219-904a-4f13-8472-e5ce851b983e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:40.883952 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96102219-904a-4f13-8472-e5ce851b983e admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:40.884377 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96102219-904a-4f13-8472-e5ce851b983e admin admin] PATH_INFO: `/v3/projects/f0ef8aa91db840a29bf730454ee30a06` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:40.886042 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96102219-904a-4f13-8472-e5ce851b983e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0ef8aa91db840a29bf730454ee30a06)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:40.888405 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96102219-904a-4f13-8472-e5ce851b983e admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:40.944231 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-96102219-904a-4f13-8472-e5ce851b983e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0ef8aa91db840a29bf730454ee30a06', 'request_id': 'req-96102219-904a-4f13-8472-e5ce851b983e'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:40.954353 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d5e23a5b-fbf9-4289-8c64-edd1760187b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '530d2cfb58d84be9a14120f02955c639'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:40.956073 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d5e23a5b-fbf9-4289-8c64-edd1760187b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '530d2cfb58d84be9a14120f02955c639'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:40.956461 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-96102219-904a-4f13-8472-e5ce851b983e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0ef8aa91db840a29bf730454ee30a06', 'request_id': 'req-96102219-904a-4f13-8472-e5ce851b983e'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:40.965178 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d5e23a5b-fbf9-4289-8c64-edd1760187b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '530d2cfb58d84be9a14120f02955c639'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:40.968641 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-96102219-904a-4f13-8472-e5ce851b983e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0ef8aa91db840a29bf730454ee30a06', 'request_id': 'req-96102219-904a-4f13-8472-e5ce851b983e'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:40.980653 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d5e23a5b-fbf9-4289-8c64-edd1760187b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '530d2cfb58d84be9a14120f02955c639'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:40.986382 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-96102219-904a-4f13-8472-e5ce851b983e admin admin] The token cache is being invalidate because project f0ef8aa91db840a29bf730454ee30a06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 08:45:40.986953 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-96102219-904a-4f13-8472-e5ce851b983e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:40.988091 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-96102219-904a-4f13-8472-e5ce851b983e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:40.990358 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d5e23a5b-fbf9-4289-8c64-edd1760187b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '530d2cfb58d84be9a14120f02955c639'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:40.994598 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 791/1586] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 08:45:40 2022] DELETE /identity/v3/projects/f0ef8aa91db840a29bf730454ee30a06 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 08:45:40.995150 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96102219-904a-4f13-8472-e5ce851b983e admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:41.000032 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-96102219-904a-4f13-8472-e5ce851b983e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:41.006764 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 796/1587] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 08:45:40 2022] DELETE /identity/v3/users/530d2cfb58d84be9a14120f02955c639 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:41.010348 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6392abe-bcfb-4589-8a4a-75d0e4ddb5a7 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:41.010348 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6392abe-bcfb-4589-8a4a-75d0e4ddb5a7 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:41.010348 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6392abe-bcfb-4589-8a4a-75d0e4ddb5a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:41.030230 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-a6392abe-bcfb-4589-8a4a-75d0e4ddb5a7 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:41.031508 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-a6392abe-bcfb-4589-8a4a-75d0e4ddb5a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:41.088547 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96102219-904a-4f13-8472-e5ce851b983e admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:41.089672 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:41.102623 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 797/1588] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:45:41.104335 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b44c2-57d0-4c7a-8319-4d761b45c0dd None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:41.104627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b44c2-57d0-4c7a-8319-4d761b45c0dd None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:41.104904 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b44c2-57d0-4c7a-8319-4d761b45c0dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:41.140872 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:41.144956 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-861b44c2-57d0-4c7a-8319-4d761b45c0dd None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:41.146911 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-861b44c2-57d0-4c7a-8319-4d761b45c0dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:41.161563 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 798/1589] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:45:41.163996 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2a2fc-0b9d-4e9f-86d6-df479d2d5f4c None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:41.164382 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2a2fc-0b9d-4e9f-86d6-df479d2d5f4c None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:41.164954 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2a2fc-0b9d-4e9f-86d6-df479d2d5f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:41.175917 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d3d7f1-a1dd-4fe7-80c2-d85c0d150b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:41.176576 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3d7f1-a1dd-4fe7-80c2-d85c0d150b96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:41.176857 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3d7f1-a1dd-4fe7-80c2-d85c0d150b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:41.177131 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3d7f1-a1dd-4fe7-80c2-d85c0d150b96 admin admin] PATH_INFO: `/v3/users/9b70c56a09a6493282e7dc6196131949` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:41.178312 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d3d7f1-a1dd-4fe7-80c2-d85c0d150b96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b70c56a09a6493282e7dc6196131949)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:41.181545 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d3d7f1-a1dd-4fe7-80c2-d85c0d150b96 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:41.191068 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-71e2a2fc-0b9d-4e9f-86d6-df479d2d5f4c None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:41.192293 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-71e2a2fc-0b9d-4e9f-86d6-df479d2d5f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:41.206473 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 799/1590] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:45:41.206473 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e2a2fc-0b9d-4e9f-86d6-df479d2d5f4c None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:41.208334 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-71e2a2fc-0b9d-4e9f-86d6-df479d2d5f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:41.240735 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-b1d3d7f1-a1dd-4fe7-80c2-d85c0d150b96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b70c56a09a6493282e7dc6196131949'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:41.244251 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-b1d3d7f1-a1dd-4fe7-80c2-d85c0d150b96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b70c56a09a6493282e7dc6196131949'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:41.258829 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-b1d3d7f1-a1dd-4fe7-80c2-d85c0d150b96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b70c56a09a6493282e7dc6196131949'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:41.269481 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-b1d3d7f1-a1dd-4fe7-80c2-d85c0d150b96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b70c56a09a6493282e7dc6196131949'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:41.270808 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-b1d3d7f1-a1dd-4fe7-80c2-d85c0d150b96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b70c56a09a6493282e7dc6196131949'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:41.279297 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e2a2fc-0b9d-4e9f-86d6-df479d2d5f4c None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:41.283364 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7694c3-783b-4d1c-9a3a-2cf5801c9677 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:41.283837 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7694c3-783b-4d1c-9a3a-2cf5801c9677 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:41.284154 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7694c3-783b-4d1c-9a3a-2cf5801c9677 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:41.284450 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7694c3-783b-4d1c-9a3a-2cf5801c9677 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:41.297733 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-6e7694c3-783b-4d1c-9a3a-2cf5801c9677 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:41.297939 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 792/1591] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 08:45:40 2022] DELETE /identity/v3/users/9b70c56a09a6493282e7dc6196131949 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:41.297939 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d3d7f1-a1dd-4fe7-80c2-d85c0d150b96 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:41.297939 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-b1d3d7f1-a1dd-4fe7-80c2-d85c0d150b96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:41.307752 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7694c3-783b-4d1c-9a3a-2cf5801c9677 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:41.308191 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7694c3-783b-4d1c-9a3a-2cf5801c9677 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:41.319365 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 800/1592] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:45:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:45:41.320928 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a97107-bb31-4620-9915-66d2cdca7977 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:41.321315 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a97107-bb31-4620-9915-66d2cdca7977 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:41.321706 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a97107-bb31-4620-9915-66d2cdca7977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:41.351447 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-09a97107-bb31-4620-9915-66d2cdca7977 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:41.352684 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-09a97107-bb31-4620-9915-66d2cdca7977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:41.396247 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d3d7f1-a1dd-4fe7-80c2-d85c0d150b96 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:41.408215 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b3584f-a741-4371-9b3b-cbb8372d7263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:41.409068 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b3584f-a741-4371-9b3b-cbb8372d7263 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:41.409445 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b3584f-a741-4371-9b3b-cbb8372d7263 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:41.409758 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b3584f-a741-4371-9b3b-cbb8372d7263 admin admin] PATH_INFO: `/v3/domains` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:41.410571 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b3584f-a741-4371-9b3b-cbb8372d7263 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:45:41.410887 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b3584f-a741-4371-9b3b-cbb8372d7263 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:41.413692 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b3584f-a741-4371-9b3b-cbb8372d7263 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:41.430319 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 801/1593] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:45:41.434695 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 793/1594] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 08:45:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:45:41.454955 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a97107-bb31-4620-9915-66d2cdca7977 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:41.455556 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b3584f-a741-4371-9b3b-cbb8372d7263 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:41.458823 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-88b3584f-a741-4371-9b3b-cbb8372d7263 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:41.467753 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a97107-bb31-4620-9915-66d2cdca7977 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:41.481174 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eb0508-81a2-4f20-90ec-9815eb1d8a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:41.482446 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb0508-81a2-4f20-90ec-9815eb1d8a37 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:41.482895 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb0508-81a2-4f20-90ec-9815eb1d8a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:41.483311 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb0508-81a2-4f20-90ec-9815eb1d8a37 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:41.484567 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eb0508-81a2-4f20-90ec-9815eb1d8a37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:41.486693 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eb0508-81a2-4f20-90ec-9815eb1d8a37 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:41.537871 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 802/1595] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 08:45:41 2022] POST /identity/v3/projects => generated 385 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:45:41.544511 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eb0508-81a2-4f20-90ec-9815eb1d8a37 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:41.585999 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b3584f-a741-4371-9b3b-cbb8372d7263 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:41.593211 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:41.631332 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eb0508-81a2-4f20-90ec-9815eb1d8a37 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:41.637640 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e30935-6df8-4bb1-868f-1460c59c2ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:41.638368 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e30935-6df8-4bb1-868f-1460c59c2ba8 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:41.638714 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e30935-6df8-4bb1-868f-1460c59c2ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:41.639057 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e30935-6df8-4bb1-868f-1460c59c2ba8 admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:41.639760 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e30935-6df8-4bb1-868f-1460c59c2ba8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:41.642658 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e30935-6df8-4bb1-868f-1460c59c2ba8 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:41.657834 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8127b7-2d54-46aa-b588-5b21c5a789bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:41.658718 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8127b7-2d54-46aa-b588-5b21c5a789bb service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:41.659148 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8127b7-2d54-46aa-b588-5b21c5a789bb service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:41.659569 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8127b7-2d54-46aa-b588-5b21c5a789bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:41.665835 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:41.679223 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8127b7-2d54-46aa-b588-5b21c5a789bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:41.681555 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8127b7-2d54-46aa-b588-5b21c5a789bb service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:41.684966 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 803/1596] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:45:41 2022] POST /identity/v3/users => generated 293 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:45:41.689302 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e30935-6df8-4bb1-868f-1460c59c2ba8 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:41.699287 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 794/1597] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:45:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:45:41.704608 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e30935-6df8-4bb1-868f-1460c59c2ba8 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:41.711164 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8b085b-92e4-47a2-9f13-fc8077f2f378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:41.711973 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8b085b-92e4-47a2-9f13-fc8077f2f378 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:41.712457 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8b085b-92e4-47a2-9f13-fc8077f2f378 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:41.712942 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8b085b-92e4-47a2-9f13-fc8077f2f378 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:41.713819 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8b085b-92e4-47a2-9f13-fc8077f2f378 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:41.716468 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8b085b-92e4-47a2-9f13-fc8077f2f378 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:41.735488 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 804/1598] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:45:41 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:45:41.738394 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8127b7-2d54-46aa-b588-5b21c5a789bb service neutron] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:41.749197 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8127b7-2d54-46aa-b588-5b21c5a789bb service neutron] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:41.754981 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510ce800-fbc8-431d-85a7-d7db06839492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:41.755600 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ce800-fbc8-431d-85a7-d7db06839492 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:41.755909 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ce800-fbc8-431d-85a7-d7db06839492 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:41.756178 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ce800-fbc8-431d-85a7-d7db06839492 admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:41.757034 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510ce800-fbc8-431d-85a7-d7db06839492 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:41.758905 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510ce800-fbc8-431d-85a7-d7db06839492 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:41.771423 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 795/1599] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:45:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:45:41.776857 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8b085b-92e4-47a2-9f13-fc8077f2f378 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:41.796610 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8b085b-92e4-47a2-9f13-fc8077f2f378 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:41.802335 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05754f3f-f0b9-4f33-bccd-839ae7d9959c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:41.803144 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05754f3f-f0b9-4f33-bccd-839ae7d9959c admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:41.803431 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05754f3f-f0b9-4f33-bccd-839ae7d9959c admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:41.803897 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05754f3f-f0b9-4f33-bccd-839ae7d9959c admin admin] PATH_INFO: `/v3/projects/46cf08ad11e24e41b1d8c03a518e3a85/users/1d586f92abde4b32bc2ad0ff46923744/roles/3248f0268738489b883041851d681bb7` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:41.823942 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05754f3f-f0b9-4f33-bccd-839ae7d9959c admin admin] RBAC: Authorizing `identity:create_grant(project_id=46cf08ad11e24e41b1d8c03a518e3a85, user_id=1d586f92abde4b32bc2ad0ff46923744, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:41.829261 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05754f3f-f0b9-4f33-bccd-839ae7d9959c admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:41.863652 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 805/1600] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:45:41 2022] PUT /identity/v3/projects/46cf08ad11e24e41b1d8c03a518e3a85/users/1d586f92abde4b32bc2ad0ff46923744/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 08:45:41.875437 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eed5fe9-79d3-4757-9b7b-e3bc32e37077 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:41.876502 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eed5fe9-79d3-4757-9b7b-e3bc32e37077 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:41.876884 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eed5fe9-79d3-4757-9b7b-e3bc32e37077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:41.911429 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05754f3f-f0b9-4f33-bccd-839ae7d9959c admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:41.913464 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-05754f3f-f0b9-4f33-bccd-839ae7d9959c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:41.920622 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-1eed5fe9-79d3-4757-9b7b-e3bc32e37077 None None] MFA Rules not processed for user `1d586f92abde4b32bc2ad0ff46923744`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:41.922137 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-1eed5fe9-79d3-4757-9b7b-e3bc32e37077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:42.002003 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05754f3f-f0b9-4f33-bccd-839ae7d9959c admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:42.021982 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adab3e79-eeef-46fa-8dde-f24ea8ee126c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:42.024027 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab3e79-eeef-46fa-8dde-f24ea8ee126c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:42.025347 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab3e79-eeef-46fa-8dde-f24ea8ee126c admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:42.026288 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab3e79-eeef-46fa-8dde-f24ea8ee126c admin admin] PATH_INFO: `/v3/projects/c005fb5fbbf3481d832e40f3962b4cb5` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:42.028210 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adab3e79-eeef-46fa-8dde-f24ea8ee126c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c005fb5fbbf3481d832e40f3962b4cb5)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:42.031595 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adab3e79-eeef-46fa-8dde-f24ea8ee126c admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:42.040789 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 796/1601] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:45:42.045064 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eed5fe9-79d3-4757-9b7b-e3bc32e37077 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:42.056707 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eed5fe9-79d3-4757-9b7b-e3bc32e37077 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:42.061740 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e1e45a-550b-43be-95f9-e6793a1ce1ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:42.062439 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e1e45a-550b-43be-95f9-e6793a1ce1ea admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:42.062794 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e1e45a-550b-43be-95f9-e6793a1ce1ea admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:42.063717 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e1e45a-550b-43be-95f9-e6793a1ce1ea admin admin] PATH_INFO: `/v3/projects` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:42.064435 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e1e45a-550b-43be-95f9-e6793a1ce1ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:42.066615 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e1e45a-550b-43be-95f9-e6793a1ce1ea admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:42.097338 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-adab3e79-eeef-46fa-8dde-f24ea8ee126c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c005fb5fbbf3481d832e40f3962b4cb5', 'request_id': 'req-adab3e79-eeef-46fa-8dde-f24ea8ee126c'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:42.101105 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 797/1602] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 08:45:42 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:45:42.105089 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e1e45a-550b-43be-95f9-e6793a1ce1ea admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:42.105953 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-adab3e79-eeef-46fa-8dde-f24ea8ee126c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c005fb5fbbf3481d832e40f3962b4cb5', 'request_id': 'req-adab3e79-eeef-46fa-8dde-f24ea8ee126c'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:42.120262 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-adab3e79-eeef-46fa-8dde-f24ea8ee126c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c005fb5fbbf3481d832e40f3962b4cb5', 'request_id': 'req-adab3e79-eeef-46fa-8dde-f24ea8ee126c'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:42.131906 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-adab3e79-eeef-46fa-8dde-f24ea8ee126c admin admin] The token cache is being invalidate because project c005fb5fbbf3481d832e40f3962b4cb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 08:45:42.131906 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-adab3e79-eeef-46fa-8dde-f24ea8ee126c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:42.132629 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-adab3e79-eeef-46fa-8dde-f24ea8ee126c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:42.134225 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 806/1603] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 08:45:41 2022] DELETE /identity/v3/projects/c005fb5fbbf3481d832e40f3962b4cb5 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 08:45:42.138092 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adab3e79-eeef-46fa-8dde-f24ea8ee126c admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:42.139417 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-adab3e79-eeef-46fa-8dde-f24ea8ee126c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:42.187865 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e1e45a-550b-43be-95f9-e6793a1ce1ea admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:42.193519 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d235b1a-492a-458d-ad28-5c5023f7802f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:42.194246 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d235b1a-492a-458d-ad28-5c5023f7802f admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:42.194633 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d235b1a-492a-458d-ad28-5c5023f7802f admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:42.194985 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d235b1a-492a-458d-ad28-5c5023f7802f admin admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:42.195694 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d235b1a-492a-458d-ad28-5c5023f7802f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:42.197765 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d235b1a-492a-458d-ad28-5c5023f7802f admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:42.210720 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:42.216939 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:42.227018 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 798/1604] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:45:42 2022] POST /identity/v3/users => generated 299 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:45:42.228358 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d235b1a-492a-458d-ad28-5c5023f7802f admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:42.230313 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-2d235b1a-492a-458d-ad28-5c5023f7802f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:42.232112 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adab3e79-eeef-46fa-8dde-f24ea8ee126c admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:42.238987 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9459f2-b53e-4b24-a178-582c1d8a77bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:42.240016 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9459f2-b53e-4b24-a178-582c1d8a77bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:42.240397 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9459f2-b53e-4b24-a178-582c1d8a77bf admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:42.240793 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9459f2-b53e-4b24-a178-582c1d8a77bf admin admin] PATH_INFO: `/v3/projects/c3311f1ce9094338be6b41b73ae00f2b` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:42.242207 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9459f2-b53e-4b24-a178-582c1d8a77bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3311f1ce9094338be6b41b73ae00f2b)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:42.243532 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d235b1a-492a-458d-ad28-5c5023f7802f admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:42.244282 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9459f2-b53e-4b24-a178-582c1d8a77bf admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:42.248627 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8c02fb-c088-4a77-81df-48c461418e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:42.249267 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c02fb-c088-4a77-81df-48c461418e33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:42.249551 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c02fb-c088-4a77-81df-48c461418e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:42.249859 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c02fb-c088-4a77-81df-48c461418e33 admin admin] PATH_INFO: `/v3/users/25b0cf8063774ebf8ef11d318a12147b` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:42.250989 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8c02fb-c088-4a77-81df-48c461418e33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25b0cf8063774ebf8ef11d318a12147b)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:42.253388 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8c02fb-c088-4a77-81df-48c461418e33 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:42.298204 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-3b9459f2-b53e-4b24-a178-582c1d8a77bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3311f1ce9094338be6b41b73ae00f2b', 'request_id': 'req-3b9459f2-b53e-4b24-a178-582c1d8a77bf'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:42.305906 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-3b9459f2-b53e-4b24-a178-582c1d8a77bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3311f1ce9094338be6b41b73ae00f2b', 'request_id': 'req-3b9459f2-b53e-4b24-a178-582c1d8a77bf'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:42.308479 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-1b8c02fb-c088-4a77-81df-48c461418e33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25b0cf8063774ebf8ef11d318a12147b'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:42.310183 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-1b8c02fb-c088-4a77-81df-48c461418e33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25b0cf8063774ebf8ef11d318a12147b'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:42.317487 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-3b9459f2-b53e-4b24-a178-582c1d8a77bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3311f1ce9094338be6b41b73ae00f2b', 'request_id': 'req-3b9459f2-b53e-4b24-a178-582c1d8a77bf'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:42.327948 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-1b8c02fb-c088-4a77-81df-48c461418e33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25b0cf8063774ebf8ef11d318a12147b'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:42.332746 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-3b9459f2-b53e-4b24-a178-582c1d8a77bf admin admin] The token cache is being invalidate because project c3311f1ce9094338be6b41b73ae00f2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 08:45:42.333247 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-3b9459f2-b53e-4b24-a178-582c1d8a77bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:42.335023 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-3b9459f2-b53e-4b24-a178-582c1d8a77bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:42.336294 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 807/1605] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 08:45:42 2022] DELETE /identity/v3/projects/c3311f1ce9094338be6b41b73ae00f2b => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:42.337166 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9459f2-b53e-4b24-a178-582c1d8a77bf admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:42.338599 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-1b8c02fb-c088-4a77-81df-48c461418e33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25b0cf8063774ebf8ef11d318a12147b'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:42.338731 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-3b9459f2-b53e-4b24-a178-582c1d8a77bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:42.340392 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-1b8c02fb-c088-4a77-81df-48c461418e33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25b0cf8063774ebf8ef11d318a12147b'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:42.365146 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 799/1606] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 08:45:42 2022] DELETE /identity/v3/users/25b0cf8063774ebf8ef11d318a12147b => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:42.366894 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8c02fb-c088-4a77-81df-48c461418e33 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:42.368590 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-1b8c02fb-c088-4a77-81df-48c461418e33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:42.434743 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9459f2-b53e-4b24-a178-582c1d8a77bf admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:42.492685 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8c02fb-c088-4a77-81df-48c461418e33 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:42.499285 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c913584-4817-4324-a4f4-a1061dd40f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:42.500061 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c913584-4817-4324-a4f4-a1061dd40f6f admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:42.500398 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c913584-4817-4324-a4f4-a1061dd40f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:42.500645 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4f9d1a-8d25-41ce-ad2a-429b0a985811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:42.500889 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c913584-4817-4324-a4f4-a1061dd40f6f admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:42.501631 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c913584-4817-4324-a4f4-a1061dd40f6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:42.502071 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4f9d1a-8d25-41ce-ad2a-429b0a985811 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:42.502434 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4f9d1a-8d25-41ce-ad2a-429b0a985811 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:42.502966 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4f9d1a-8d25-41ce-ad2a-429b0a985811 admin admin] PATH_INFO: `/v3/users/fa7597f6a47c4749b86528ef2a1faf48` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:42.503522 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c913584-4817-4324-a4f4-a1061dd40f6f admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:42.504709 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4f9d1a-8d25-41ce-ad2a-429b0a985811 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa7597f6a47c4749b86528ef2a1faf48)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:42.510588 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4f9d1a-8d25-41ce-ad2a-429b0a985811 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:42.515834 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 808/1607] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:45:42 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:45:42.520296 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c913584-4817-4324-a4f4-a1061dd40f6f admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:42.533058 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c913584-4817-4324-a4f4-a1061dd40f6f admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:42.543042 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1b7dce-2546-4f75-affe-c2acd7f6ee02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:42.544983 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b7dce-2546-4f75-affe-c2acd7f6ee02 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:42.546088 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b7dce-2546-4f75-affe-c2acd7f6ee02 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:42.546516 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b7dce-2546-4f75-affe-c2acd7f6ee02 admin admin] PATH_INFO: `/v3/projects/41cf643468b845b38413cc12216b11fd/users/b694624376d84d0caa7427ca5c2226ff/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:42.571004 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1b7dce-2546-4f75-affe-c2acd7f6ee02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41cf643468b845b38413cc12216b11fd, user_id=b694624376d84d0caa7427ca5c2226ff, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:42.575007 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1b7dce-2546-4f75-affe-c2acd7f6ee02 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:42.590446 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-7a4f9d1a-8d25-41ce-ad2a-429b0a985811 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa7597f6a47c4749b86528ef2a1faf48'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:42.591214 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-7a4f9d1a-8d25-41ce-ad2a-429b0a985811 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa7597f6a47c4749b86528ef2a1faf48'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:42.592164 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:42.598120 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 809/1608] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:45:42 2022] PUT /identity/v3/projects/41cf643468b845b38413cc12216b11fd/users/b694624376d84d0caa7427ca5c2226ff/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:42.604188 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1b7dce-2546-4f75-affe-c2acd7f6ee02 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:42.608422 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-bc1b7dce-2546-4f75-affe-c2acd7f6ee02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:42.614205 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-7a4f9d1a-8d25-41ce-ad2a-429b0a985811 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa7597f6a47c4749b86528ef2a1faf48'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:42.626112 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-7a4f9d1a-8d25-41ce-ad2a-429b0a985811 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa7597f6a47c4749b86528ef2a1faf48'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:42.626522 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-7a4f9d1a-8d25-41ce-ad2a-429b0a985811 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa7597f6a47c4749b86528ef2a1faf48'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:42.628777 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:42.647471 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 800/1609] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 08:45:42 2022] DELETE /identity/v3/users/fa7597f6a47c4749b86528ef2a1faf48 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:42.706948 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1b7dce-2546-4f75-affe-c2acd7f6ee02 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:42.782939 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a45d64f-77f6-4384-934a-e10c9c3b20be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:42.784026 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a45d64f-77f6-4384-934a-e10c9c3b20be admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:42.784726 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a45d64f-77f6-4384-934a-e10c9c3b20be admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:42.785140 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a45d64f-77f6-4384-934a-e10c9c3b20be admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:42.786155 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a45d64f-77f6-4384-934a-e10c9c3b20be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:42.790207 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a45d64f-77f6-4384-934a-e10c9c3b20be admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:42.803272 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 810/1610] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:45:42 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:45:42.804435 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4f9d1a-8d25-41ce-ad2a-429b0a985811 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:42.806857 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-7a4f9d1a-8d25-41ce-ad2a-429b0a985811 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:42.808141 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a45d64f-77f6-4384-934a-e10c9c3b20be admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:42.819612 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a45d64f-77f6-4384-934a-e10c9c3b20be admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:42.825100 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ad845e-cba5-4580-bf0f-d226137ab11e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:42.826638 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad845e-cba5-4580-bf0f-d226137ab11e admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:42.826931 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad845e-cba5-4580-bf0f-d226137ab11e admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:42.827214 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad845e-cba5-4580-bf0f-d226137ab11e admin admin] PATH_INFO: `/v3/projects/41cf643468b845b38413cc12216b11fd/users/b694624376d84d0caa7427ca5c2226ff/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:42.829987 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ad845e-cba5-4580-bf0f-d226137ab11e admin admin] RBAC: Authorizing `identity:create_grant(project_id=41cf643468b845b38413cc12216b11fd, user_id=b694624376d84d0caa7427ca5c2226ff, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:42.831372 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4f9d1a-8d25-41ce-ad2a-429b0a985811 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:42.832919 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ad845e-cba5-4580-bf0f-d226137ab11e admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:42.838788 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e4341c-0e11-4ff7-b90c-cbd34ae80dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:42.839584 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e4341c-0e11-4ff7-b90c-cbd34ae80dcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:42.839995 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e4341c-0e11-4ff7-b90c-cbd34ae80dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:42.840385 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e4341c-0e11-4ff7-b90c-cbd34ae80dcf admin admin] PATH_INFO: `/v3/projects/8a84972afc3b47f7b3c5603ca539e717` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:42.841798 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e4341c-0e11-4ff7-b90c-cbd34ae80dcf admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a84972afc3b47f7b3c5603ca539e717)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:42.844374 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e4341c-0e11-4ff7-b90c-cbd34ae80dcf admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:42.845501 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 811/1611] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:45:42 2022] PUT /identity/v3/projects/41cf643468b845b38413cc12216b11fd/users/b694624376d84d0caa7427ca5c2226ff/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 08:45:42.860152 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84292f4-741c-40d9-ba71-b872ffee7e7e None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:42.860552 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84292f4-741c-40d9-ba71-b872ffee7e7e None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:42.860933 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84292f4-741c-40d9-ba71-b872ffee7e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:42.897638 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-f84292f4-741c-40d9-ba71-b872ffee7e7e None None] MFA Rules not processed for user `b694624376d84d0caa7427ca5c2226ff`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:42.898924 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-f84292f4-741c-40d9-ba71-b872ffee7e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:42.907937 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-77e4341c-0e11-4ff7-b90c-cbd34ae80dcf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a84972afc3b47f7b3c5603ca539e717', 'request_id': 'req-77e4341c-0e11-4ff7-b90c-cbd34ae80dcf'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:42.935083 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-77e4341c-0e11-4ff7-b90c-cbd34ae80dcf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a84972afc3b47f7b3c5603ca539e717', 'request_id': 'req-77e4341c-0e11-4ff7-b90c-cbd34ae80dcf'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:42.958298 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-77e4341c-0e11-4ff7-b90c-cbd34ae80dcf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a84972afc3b47f7b3c5603ca539e717', 'request_id': 'req-77e4341c-0e11-4ff7-b90c-cbd34ae80dcf'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:42.979921 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 812/1612] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:42 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:45:42.991456 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84292f4-741c-40d9-ba71-b872ffee7e7e None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:42.993087 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-f84292f4-741c-40d9-ba71-b872ffee7e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:43.008656 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-77e4341c-0e11-4ff7-b90c-cbd34ae80dcf admin admin] The token cache is being invalidate because project 8a84972afc3b47f7b3c5603ca539e717 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 08:45:43.008957 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-77e4341c-0e11-4ff7-b90c-cbd34ae80dcf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:43.009642 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-77e4341c-0e11-4ff7-b90c-cbd34ae80dcf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:43.016508 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 801/1613] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 08:45:42 2022] DELETE /identity/v3/projects/8a84972afc3b47f7b3c5603ca539e717 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:43.018000 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:43.018928 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1596d12-4a6b-48b8-8729-9a30a1c6453d None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:43.019228 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1596d12-4a6b-48b8-8729-9a30a1c6453d None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:43.019595 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1596d12-4a6b-48b8-8729-9a30a1c6453d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:43.049653 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-d1596d12-4a6b-48b8-8729-9a30a1c6453d None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:43.051273 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-d1596d12-4a6b-48b8-8729-9a30a1c6453d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:43.099255 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84292f4-741c-40d9-ba71-b872ffee7e7e None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:43.105362 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33678697-0548-4a2d-8e81-dded9a712db5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:43.115877 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33678697-0548-4a2d-8e81-dded9a712db5 service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:43.117459 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33678697-0548-4a2d-8e81-dded9a712db5 service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:43.117772 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33678697-0548-4a2d-8e81-dded9a712db5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:43.126151 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-33678697-0548-4a2d-8e81-dded9a712db5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:43.151464 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33678697-0548-4a2d-8e81-dded9a712db5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:43.153080 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33678697-0548-4a2d-8e81-dded9a712db5 service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:43.170652 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 802/1614] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 08:45:43.170849 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 813/1615] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:45:42 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 08:45:43.173944 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f82cbf1-f2c0-4749-b66c-81e7e29694f7 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:43.173944 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f82cbf1-f2c0-4749-b66c-81e7e29694f7 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:43.173944 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f82cbf1-f2c0-4749-b66c-81e7e29694f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:43.221216 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-4f82cbf1-f2c0-4749-b66c-81e7e29694f7 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:43.222540 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-4f82cbf1-f2c0-4749-b66c-81e7e29694f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:43.232465 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 803/1616] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:45:43.239027 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33678697-0548-4a2d-8e81-dded9a712db5 service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:43.261946 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33678697-0548-4a2d-8e81-dded9a712db5 service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:43.262368 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f82cbf1-f2c0-4749-b66c-81e7e29694f7 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:43.265984 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-4f82cbf1-f2c0-4749-b66c-81e7e29694f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:43.278590 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a8c8c5-acce-4a7e-8561-9f8636815115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:43.278590 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a8c8c5-acce-4a7e-8561-9f8636815115 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:43.278590 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a8c8c5-acce-4a7e-8561-9f8636815115 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:43.278590 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a8c8c5-acce-4a7e-8561-9f8636815115 admin admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:43.279084 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a8c8c5-acce-4a7e-8561-9f8636815115 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:45:43.279084 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a8c8c5-acce-4a7e-8561-9f8636815115 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:43.280538 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a8c8c5-acce-4a7e-8561-9f8636815115 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:43.302467 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f82cbf1-f2c0-4749-b66c-81e7e29694f7 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:43.316260 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d531be-4481-492e-ac1f-0f7c41c844cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:43.317200 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d531be-4481-492e-ac1f-0f7c41c844cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:43.317589 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d531be-4481-492e-ac1f-0f7c41c844cb admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:43.318497 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d531be-4481-492e-ac1f-0f7c41c844cb admin admin] PATH_INFO: `/v3/projects/0602f0f391444439a1a1d36a628ed1c1` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:43.323537 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 814/1617] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 08:45:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:45:43.326887 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d531be-4481-492e-ac1f-0f7c41c844cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=0602f0f391444439a1a1d36a628ed1c1)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:43.330317 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a8c8c5-acce-4a7e-8561-9f8636815115 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:43.333352 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-a1a8c8c5-acce-4a7e-8561-9f8636815115 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:43.351109 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d531be-4481-492e-ac1f-0f7c41c844cb admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:43.351378 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a8c8c5-acce-4a7e-8561-9f8636815115 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:43.363681 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7878e760-6b9e-4860-8b08-16c029951422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:43.364342 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878e760-6b9e-4860-8b08-16c029951422 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:43.364633 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878e760-6b9e-4860-8b08-16c029951422 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:43.364917 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878e760-6b9e-4860-8b08-16c029951422 admin admin] PATH_INFO: `/v3/users/1d586f92abde4b32bc2ad0ff46923744` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:43.372080 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7878e760-6b9e-4860-8b08-16c029951422 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d586f92abde4b32bc2ad0ff46923744)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:43.375056 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7878e760-6b9e-4860-8b08-16c029951422 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:43.420556 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-13d531be-4481-492e-ac1f-0f7c41c844cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0602f0f391444439a1a1d36a628ed1c1', 'request_id': 'req-13d531be-4481-492e-ac1f-0f7c41c844cb'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:43.428497 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-13d531be-4481-492e-ac1f-0f7c41c844cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0602f0f391444439a1a1d36a628ed1c1', 'request_id': 'req-13d531be-4481-492e-ac1f-0f7c41c844cb'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:43.437117 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-13d531be-4481-492e-ac1f-0f7c41c844cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0602f0f391444439a1a1d36a628ed1c1', 'request_id': 'req-13d531be-4481-492e-ac1f-0f7c41c844cb'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:43.438538 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7878e760-6b9e-4860-8b08-16c029951422 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d586f92abde4b32bc2ad0ff46923744'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:43.438796 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:43.439897 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7878e760-6b9e-4860-8b08-16c029951422 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d586f92abde4b32bc2ad0ff46923744'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:43.449962 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7878e760-6b9e-4860-8b08-16c029951422 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d586f92abde4b32bc2ad0ff46923744'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:43.451867 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-13d531be-4481-492e-ac1f-0f7c41c844cb admin admin] The token cache is being invalidate because project 0602f0f391444439a1a1d36a628ed1c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 08:45:43.452378 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-13d531be-4481-492e-ac1f-0f7c41c844cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:43.453779 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-13d531be-4481-492e-ac1f-0f7c41c844cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:43.455549 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 804/1618] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 08:45:43 2022] DELETE /identity/v3/projects/0602f0f391444439a1a1d36a628ed1c1 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:43.460536 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d531be-4481-492e-ac1f-0f7c41c844cb admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:43.462044 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-13d531be-4481-492e-ac1f-0f7c41c844cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:43.469204 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7878e760-6b9e-4860-8b08-16c029951422 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d586f92abde4b32bc2ad0ff46923744'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:43.479202 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7878e760-6b9e-4860-8b08-16c029951422 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d586f92abde4b32bc2ad0ff46923744'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:43.489280 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 815/1619] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 08:45:43 2022] DELETE /identity/v3/users/1d586f92abde4b32bc2ad0ff46923744 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:43.489280 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26b805e-1a7c-4a0f-b655-01fe385f8729 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:43.489280 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26b805e-1a7c-4a0f-b655-01fe385f8729 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:43.489280 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26b805e-1a7c-4a0f-b655-01fe385f8729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:43.514305 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-a26b805e-1a7c-4a0f-b655-01fe385f8729 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:43.515902 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-a26b805e-1a7c-4a0f-b655-01fe385f8729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:43.557381 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d531be-4481-492e-ac1f-0f7c41c844cb admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:43.603252 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 816/1620] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:45:43.611236 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef0677-7550-42aa-9f7d-611fc5ffffb4 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:43.611236 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef0677-7550-42aa-9f7d-611fc5ffffb4 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:43.611236 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef0677-7550-42aa-9f7d-611fc5ffffb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:43.628521 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-8cef0677-7550-42aa-9f7d-611fc5ffffb4 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:43.630219 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-8cef0677-7550-42aa-9f7d-611fc5ffffb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:43.633440 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be05270a-fc85-49c6-ac68-c695114e0798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:43.634220 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be05270a-fc85-49c6-ac68-c695114e0798 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:43.634575 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be05270a-fc85-49c6-ac68-c695114e0798 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:43.634920 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be05270a-fc85-49c6-ac68-c695114e0798 admin admin] PATH_INFO: `/v3/projects` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:43.635662 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be05270a-fc85-49c6-ac68-c695114e0798 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:43.637760 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be05270a-fc85-49c6-ac68-c695114e0798 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:43.640397 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 817/1621] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:45:43.641889 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a311b50-bb38-4301-8ef3-cde33bbc630c None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:43.642178 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a311b50-bb38-4301-8ef3-cde33bbc630c None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:43.642608 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a311b50-bb38-4301-8ef3-cde33bbc630c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:43.666997 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-4a311b50-bb38-4301-8ef3-cde33bbc630c None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:43.668675 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-4a311b50-bb38-4301-8ef3-cde33bbc630c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:43.678761 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 805/1622] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 08:45:43 2022] POST /identity/v3/projects => generated 383 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:45:43.678761 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be05270a-fc85-49c6-ac68-c695114e0798 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:43.683791 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 818/1623] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:45:43.683956 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-be05270a-fc85-49c6-ac68-c695114e0798 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:43.684082 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a311b50-bb38-4301-8ef3-cde33bbc630c None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:43.690652 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:43.707506 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a311b50-bb38-4301-8ef3-cde33bbc630c None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:43.758807 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be05270a-fc85-49c6-ac68-c695114e0798 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:43.763689 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee79f7b2-c365-4b72-b6d2-7f9b806f52d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:43.764374 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee79f7b2-c365-4b72-b6d2-7f9b806f52d9 service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:43.764653 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee79f7b2-c365-4b72-b6d2-7f9b806f52d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:43.764935 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee79f7b2-c365-4b72-b6d2-7f9b806f52d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:43.815562 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc15c347-4d07-45c0-b85c-7fb310621a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:43.816465 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15c347-4d07-45c0-b85c-7fb310621a60 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:43.816753 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15c347-4d07-45c0-b85c-7fb310621a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:43.817347 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15c347-4d07-45c0-b85c-7fb310621a60 admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:43.818037 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc15c347-4d07-45c0-b85c-7fb310621a60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:43.821112 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc15c347-4d07-45c0-b85c-7fb310621a60 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:43.829731 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:43.857572 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 819/1624] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:45:43 2022] POST /identity/v3/users => generated 292 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:45:43.858363 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc15c347-4d07-45c0-b85c-7fb310621a60 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:43.871104 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc15c347-4d07-45c0-b85c-7fb310621a60 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:43.876690 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1771f9b8-9dd1-4197-ab9f-d09dea5f17de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:43.877309 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1771f9b8-9dd1-4197-ab9f-d09dea5f17de admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:43.877744 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1771f9b8-9dd1-4197-ab9f-d09dea5f17de admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:43.878035 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1771f9b8-9dd1-4197-ab9f-d09dea5f17de admin admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:43.878858 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1771f9b8-9dd1-4197-ab9f-d09dea5f17de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:45:43.879157 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1771f9b8-9dd1-4197-ab9f-d09dea5f17de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:43.880455 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee79f7b2-c365-4b72-b6d2-7f9b806f52d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:43.881090 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1771f9b8-9dd1-4197-ab9f-d09dea5f17de admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:43.882568 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee79f7b2-c365-4b72-b6d2-7f9b806f52d9 service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:43.899253 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 806/1625] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:45:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:45:43.906802 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee79f7b2-c365-4b72-b6d2-7f9b806f52d9 service neutron] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:43.917016 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee79f7b2-c365-4b72-b6d2-7f9b806f52d9 service neutron] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:43.919139 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 820/1626] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 08:45:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:45:43.924175 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1771f9b8-9dd1-4197-ab9f-d09dea5f17de admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:43.925355 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ee3895-2544-43b1-b553-57c13b69e567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:43.926223 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ee3895-2544-43b1-b553-57c13b69e567 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:43.926884 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ee3895-2544-43b1-b553-57c13b69e567 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:43.926884 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ee3895-2544-43b1-b553-57c13b69e567 admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:43.927478 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ee3895-2544-43b1-b553-57c13b69e567 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:43.929988 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ee3895-2544-43b1-b553-57c13b69e567 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:43.938097 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1771f9b8-9dd1-4197-ab9f-d09dea5f17de admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:43.943744 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b95920-fda7-4b92-bc27-cb66ae2e911e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:43.944120 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b95920-fda7-4b92-bc27-cb66ae2e911e admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:43.945164 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b95920-fda7-4b92-bc27-cb66ae2e911e admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:43.945398 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b95920-fda7-4b92-bc27-cb66ae2e911e admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:43.946212 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b95920-fda7-4b92-bc27-cb66ae2e911e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:43.948200 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b95920-fda7-4b92-bc27-cb66ae2e911e admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:43.953395 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 807/1627] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:45:43 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:45:43.955403 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ee3895-2544-43b1-b553-57c13b69e567 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:43.974623 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ee3895-2544-43b1-b553-57c13b69e567 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:43.981766 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f518d1-3699-4045-bc0e-db149f9ca8aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:43.982661 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f518d1-3699-4045-bc0e-db149f9ca8aa admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:43.983160 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f518d1-3699-4045-bc0e-db149f9ca8aa admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:43.984014 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f518d1-3699-4045-bc0e-db149f9ca8aa admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:43.984946 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f518d1-3699-4045-bc0e-db149f9ca8aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:43.986840 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f518d1-3699-4045-bc0e-db149f9ca8aa admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:43.993922 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 821/1628] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 08:45:43 2022] POST /identity/v3/projects => generated 381 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:45:43.995413 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b95920-fda7-4b92-bc27-cb66ae2e911e admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:44.006304 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 808/1629] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:45:43 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 08:45:44.011794 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f518d1-3699-4045-bc0e-db149f9ca8aa admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:44.039261 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f518d1-3699-4045-bc0e-db149f9ca8aa admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:44.130029 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b95920-fda7-4b92-bc27-cb66ae2e911e admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:44.135739 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37675583-a8ec-481b-a8a0-ac0a4f48c1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:44.136362 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37675583-a8ec-481b-a8a0-ac0a4f48c1c9 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:44.136648 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37675583-a8ec-481b-a8a0-ac0a4f48c1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:44.136926 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37675583-a8ec-481b-a8a0-ac0a4f48c1c9 admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:44.137574 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37675583-a8ec-481b-a8a0-ac0a4f48c1c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:44.144279 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37675583-a8ec-481b-a8a0-ac0a4f48c1c9 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:44.153788 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6738b2-4674-4cbc-ab00-5feb12e0f747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:44.155884 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6738b2-4674-4cbc-ab00-5feb12e0f747 admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:44.156395 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6738b2-4674-4cbc-ab00-5feb12e0f747 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:44.157278 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6738b2-4674-4cbc-ab00-5feb12e0f747 admin admin] PATH_INFO: `/v3/projects/5347240119704ed583006c50eebc0c7c/users/3ec0aebc837d488f84806c266dbaf392/roles/3248f0268738489b883041851d681bb7` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:44.177645 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 822/1630] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:45:43 2022] POST /identity/v3/users => generated 291 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 08:45:44.179872 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37675583-a8ec-481b-a8a0-ac0a4f48c1c9 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:44.187855 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6738b2-4674-4cbc-ab00-5feb12e0f747 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5347240119704ed583006c50eebc0c7c, user_id=3ec0aebc837d488f84806c266dbaf392, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:44.191262 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37675583-a8ec-481b-a8a0-ac0a4f48c1c9 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:44.192521 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6738b2-4674-4cbc-ab00-5feb12e0f747 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:44.196719 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a294583-b8e0-4a35-afa2-6a2570b091b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:44.197391 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a294583-b8e0-4a35-afa2-6a2570b091b3 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:44.198451 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a294583-b8e0-4a35-afa2-6a2570b091b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:44.198774 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a294583-b8e0-4a35-afa2-6a2570b091b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:44.199398 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a294583-b8e0-4a35-afa2-6a2570b091b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:44.201098 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a294583-b8e0-4a35-afa2-6a2570b091b3 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:44.214297 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 823/1631] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:45:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 08:45:44.218126 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a294583-b8e0-4a35-afa2-6a2570b091b3 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:44.230265 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 809/1632] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:45:44 2022] PUT /identity/v3/projects/5347240119704ed583006c50eebc0c7c/users/3ec0aebc837d488f84806c266dbaf392/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:44.231935 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a294583-b8e0-4a35-afa2-6a2570b091b3 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:44.232269 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e6a93c-2a49-44c0-b823-6ad9268e265a None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:44.232666 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e6a93c-2a49-44c0-b823-6ad9268e265a None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:44.232957 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e6a93c-2a49-44c0-b823-6ad9268e265a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:44.271978 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-55e6a93c-2a49-44c0-b823-6ad9268e265a None None] MFA Rules not processed for user `3ec0aebc837d488f84806c266dbaf392`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:44.273252 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-55e6a93c-2a49-44c0-b823-6ad9268e265a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:44.273523 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:44.317295 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e3d372-1906-4070-bd16-5fb20e615f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:44.318268 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e3d372-1906-4070-bd16-5fb20e615f63 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:44.318651 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e3d372-1906-4070-bd16-5fb20e615f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:44.319091 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e3d372-1906-4070-bd16-5fb20e615f63 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:44.320608 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e3d372-1906-4070-bd16-5fb20e615f63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:44.322660 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e3d372-1906-4070-bd16-5fb20e615f63 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:44.336722 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 824/1633] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:45:44 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 08:45:44.341022 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e3d372-1906-4070-bd16-5fb20e615f63 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:44.365064 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e3d372-1906-4070-bd16-5fb20e615f63 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:44.376985 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9a606a-8d54-4fac-8c4f-284bad038d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:44.376985 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a606a-8d54-4fac-8c4f-284bad038d0c admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:44.376985 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a606a-8d54-4fac-8c4f-284bad038d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:44.377586 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a606a-8d54-4fac-8c4f-284bad038d0c admin admin] PATH_INFO: `/v3/projects/33a0fa812dc14fa2be10b911a914b44f/users/d1d735a17e994291ab7e03c67ca139e0/roles/3248f0268738489b883041851d681bb7` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:44.404443 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9a606a-8d54-4fac-8c4f-284bad038d0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=33a0fa812dc14fa2be10b911a914b44f, user_id=d1d735a17e994291ab7e03c67ca139e0, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:44.404793 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 810/1634] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 08:45:44.407407 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9a606a-8d54-4fac-8c4f-284bad038d0c admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:44.409906 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e6a93c-2a49-44c0-b823-6ad9268e265a None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:44.424505 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 825/1635] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:45:44 2022] PUT /identity/v3/projects/33a0fa812dc14fa2be10b911a914b44f/users/d1d735a17e994291ab7e03c67ca139e0/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 08:45:44.435371 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fddc070-3cf9-4353-822a-4de7ea2958a3 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:44.435669 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fddc070-3cf9-4353-822a-4de7ea2958a3 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:44.435958 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fddc070-3cf9-4353-822a-4de7ea2958a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:44.436079 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e6a93c-2a49-44c0-b823-6ad9268e265a None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:44.470697 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-4fddc070-3cf9-4353-822a-4de7ea2958a3 None None] MFA Rules not processed for user `d1d735a17e994291ab7e03c67ca139e0`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:44.472206 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-4fddc070-3cf9-4353-822a-4de7ea2958a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:44.473991 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:44.532750 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca9fc01-d82c-4d29-bc5f-1eb844455400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:44.533389 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca9fc01-d82c-4d29-bc5f-1eb844455400 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:44.533690 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca9fc01-d82c-4d29-bc5f-1eb844455400 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:44.533992 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca9fc01-d82c-4d29-bc5f-1eb844455400 admin admin] PATH_INFO: `/v3/projects` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:44.534627 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca9fc01-d82c-4d29-bc5f-1eb844455400 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:44.538479 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca9fc01-d82c-4d29-bc5f-1eb844455400 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:44.600540 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 811/1636] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 08:45:44 2022] POST /identity/v3/projects => generated 383 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 08:45:44.623405 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca9fc01-d82c-4d29-bc5f-1eb844455400 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:44.658258 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 826/1637] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:44 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:45:44.659227 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fddc070-3cf9-4353-822a-4de7ea2958a3 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:44.662457 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-4fddc070-3cf9-4353-822a-4de7ea2958a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:44.690512 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fddc070-3cf9-4353-822a-4de7ea2958a3 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:44.726588 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca9fc01-d82c-4d29-bc5f-1eb844455400 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:44.731903 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9254cd-00d5-4849-bac0-96e03ab3c2a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:44.732845 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9254cd-00d5-4849-bac0-96e03ab3c2a2 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:44.733450 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9254cd-00d5-4849-bac0-96e03ab3c2a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:44.734020 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9254cd-00d5-4849-bac0-96e03ab3c2a2 admin admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:44.734965 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9254cd-00d5-4849-bac0-96e03ab3c2a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:44.737112 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9254cd-00d5-4849-bac0-96e03ab3c2a2 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:44.764012 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 812/1638] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:45:44 2022] POST /identity/v3/users => generated 298 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:45:44.765882 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9254cd-00d5-4849-bac0-96e03ab3c2a2 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:44.771291 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe5be38-54ee-44bc-89f8-b25429abbce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:44.773842 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe5be38-54ee-44bc-89f8-b25429abbce9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:44.774214 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe5be38-54ee-44bc-89f8-b25429abbce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:44.774832 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe5be38-54ee-44bc-89f8-b25429abbce9 admin admin] PATH_INFO: `/v3/projects/46cf08ad11e24e41b1d8c03a518e3a85` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:44.776465 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe5be38-54ee-44bc-89f8-b25429abbce9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46cf08ad11e24e41b1d8c03a518e3a85)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:44.778519 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe5be38-54ee-44bc-89f8-b25429abbce9 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:44.778899 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9254cd-00d5-4849-bac0-96e03ab3c2a2 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:44.785841 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f421c40-c56c-4f20-aabf-a7aaabb8b757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:44.786817 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f421c40-c56c-4f20-aabf-a7aaabb8b757 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:44.787832 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f421c40-c56c-4f20-aabf-a7aaabb8b757 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:44.788422 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f421c40-c56c-4f20-aabf-a7aaabb8b757 admin admin] PATH_INFO: `/v3/projects` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:44.789917 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f421c40-c56c-4f20-aabf-a7aaabb8b757 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:44.792667 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f421c40-c56c-4f20-aabf-a7aaabb8b757 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:44.831685 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 813/1639] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 08:45:44 2022] POST /identity/v3/projects => generated 379 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:45:44.832458 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f421c40-c56c-4f20-aabf-a7aaabb8b757 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:44.843602 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:44.859506 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8fe5be38-54ee-44bc-89f8-b25429abbce9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46cf08ad11e24e41b1d8c03a518e3a85', 'request_id': 'req-8fe5be38-54ee-44bc-89f8-b25429abbce9'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:44.868810 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8fe5be38-54ee-44bc-89f8-b25429abbce9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46cf08ad11e24e41b1d8c03a518e3a85', 'request_id': 'req-8fe5be38-54ee-44bc-89f8-b25429abbce9'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:44.879597 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8fe5be38-54ee-44bc-89f8-b25429abbce9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46cf08ad11e24e41b1d8c03a518e3a85', 'request_id': 'req-8fe5be38-54ee-44bc-89f8-b25429abbce9'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:44.891464 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8fe5be38-54ee-44bc-89f8-b25429abbce9 admin admin] The token cache is being invalidate because project 46cf08ad11e24e41b1d8c03a518e3a85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 08:45:44.891754 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8fe5be38-54ee-44bc-89f8-b25429abbce9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:44.893820 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8fe5be38-54ee-44bc-89f8-b25429abbce9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:44.895774 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 827/1640] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 08:45:44 2022] DELETE /identity/v3/projects/46cf08ad11e24e41b1d8c03a518e3a85 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:44.896710 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe5be38-54ee-44bc-89f8-b25429abbce9 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:44.900290 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-8fe5be38-54ee-44bc-89f8-b25429abbce9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:44.910727 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f421c40-c56c-4f20-aabf-a7aaabb8b757 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:44.940449 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:44.978967 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe5be38-54ee-44bc-89f8-b25429abbce9 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:44.986199 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb132e14-9705-4edf-a054-4551878b70e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:44.986743 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb132e14-9705-4edf-a054-4551878b70e0 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:44.987027 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb132e14-9705-4edf-a054-4551878b70e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:44.987304 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb132e14-9705-4edf-a054-4551878b70e0 admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:44.988125 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb132e14-9705-4edf-a054-4551878b70e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:44.990517 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41776bc7-0b49-4197-9df2-8536e9e998d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:44.992237 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb132e14-9705-4edf-a054-4551878b70e0 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:44.992752 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41776bc7-0b49-4197-9df2-8536e9e998d7 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:44.993257 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41776bc7-0b49-4197-9df2-8536e9e998d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:44.993854 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41776bc7-0b49-4197-9df2-8536e9e998d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:44.994678 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41776bc7-0b49-4197-9df2-8536e9e998d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:44.997265 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41776bc7-0b49-4197-9df2-8536e9e998d7 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:45.010042 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 814/1641] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:45:44 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:45:45.011317 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41776bc7-0b49-4197-9df2-8536e9e998d7 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:45.013220 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-41776bc7-0b49-4197-9df2-8536e9e998d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:45.028033 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41776bc7-0b49-4197-9df2-8536e9e998d7 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:45.028258 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 828/1642] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:45:44 2022] POST /identity/v3/users => generated 296 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:45:45.029134 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb132e14-9705-4edf-a054-4551878b70e0 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:45.035252 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876bcad4-069f-4d7f-b083-7b622338e1e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:45.036119 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876bcad4-069f-4d7f-b083-7b622338e1e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:45.036243 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-eb132e14-9705-4edf-a054-4551878b70e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:45.036658 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876bcad4-069f-4d7f-b083-7b622338e1e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:45.037089 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876bcad4-069f-4d7f-b083-7b622338e1e3 admin admin] PATH_INFO: `/v3/users/b694624376d84d0caa7427ca5c2226ff` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:45.039145 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876bcad4-069f-4d7f-b083-7b622338e1e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b694624376d84d0caa7427ca5c2226ff)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:45.041080 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876bcad4-069f-4d7f-b083-7b622338e1e3 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:45.053527 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb132e14-9705-4edf-a054-4551878b70e0 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:45.059386 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbb2791-2416-49d3-a6ec-21fa9470847c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:45.060177 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb2791-2416-49d3-a6ec-21fa9470847c admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:45.060600 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb2791-2416-49d3-a6ec-21fa9470847c admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:45.060982 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb2791-2416-49d3-a6ec-21fa9470847c admin admin] PATH_INFO: `/v3/projects/eb68a5cb7877444ebaa2db56830d080a/users/c9d2e670b03d42ac8a58f8376647da98/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:45.081228 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbb2791-2416-49d3-a6ec-21fa9470847c admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb68a5cb7877444ebaa2db56830d080a, user_id=c9d2e670b03d42ac8a58f8376647da98, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:45.084153 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbb2791-2416-49d3-a6ec-21fa9470847c admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:45.102889 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 829/1643] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:45:45 2022] PUT /identity/v3/projects/eb68a5cb7877444ebaa2db56830d080a/users/c9d2e670b03d42ac8a58f8376647da98/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:45.103329 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbb2791-2416-49d3-a6ec-21fa9470847c admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:45.114299 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-876bcad4-069f-4d7f-b083-7b622338e1e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b694624376d84d0caa7427ca5c2226ff'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:45.115336 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-876bcad4-069f-4d7f-b083-7b622338e1e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b694624376d84d0caa7427ca5c2226ff'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:45.133421 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-876bcad4-069f-4d7f-b083-7b622338e1e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b694624376d84d0caa7427ca5c2226ff'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:45.144753 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-876bcad4-069f-4d7f-b083-7b622338e1e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b694624376d84d0caa7427ca5c2226ff'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:45.147525 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-876bcad4-069f-4d7f-b083-7b622338e1e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b694624376d84d0caa7427ca5c2226ff'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:45.168748 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 815/1644] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 08:45:45 2022] DELETE /identity/v3/users/b694624376d84d0caa7427ca5c2226ff => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:45.169704 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876bcad4-069f-4d7f-b083-7b622338e1e3 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:45.185406 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbb2791-2416-49d3-a6ec-21fa9470847c admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:45.186759 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-876bcad4-069f-4d7f-b083-7b622338e1e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:45.238567 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:45.266244 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8879ef-3f06-44ae-b40b-91527a5c3fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:45.266587 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8879ef-3f06-44ae-b40b-91527a5c3fc1 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:45.266869 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8879ef-3f06-44ae-b40b-91527a5c3fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:45.267198 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8879ef-3f06-44ae-b40b-91527a5c3fc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:45.267785 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8879ef-3f06-44ae-b40b-91527a5c3fc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:45.269372 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8879ef-3f06-44ae-b40b-91527a5c3fc1 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:45.292324 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 830/1645] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:45:45 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:45:45.293146 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876bcad4-069f-4d7f-b083-7b622338e1e3 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:45.294160 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8879ef-3f06-44ae-b40b-91527a5c3fc1 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:45.296520 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-0a8879ef-3f06-44ae-b40b-91527a5c3fc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:45.309218 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51f8c70-ec40-45bb-99d3-d8ee6fe72683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:45.310367 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f8c70-ec40-45bb-99d3-d8ee6fe72683 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:45.310657 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f8c70-ec40-45bb-99d3-d8ee6fe72683 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:45.310981 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f8c70-ec40-45bb-99d3-d8ee6fe72683 admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:45.313888 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51f8c70-ec40-45bb-99d3-d8ee6fe72683 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:45.314994 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8879ef-3f06-44ae-b40b-91527a5c3fc1 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:45.315515 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51f8c70-ec40-45bb-99d3-d8ee6fe72683 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:45.321822 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354e6b84-c20e-4d78-95f4-e7d5940a11ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:45.322514 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e6b84-c20e-4d78-95f4-e7d5940a11ef admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:45.322840 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e6b84-c20e-4d78-95f4-e7d5940a11ef admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:45.323108 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e6b84-c20e-4d78-95f4-e7d5940a11ef admin admin] PATH_INFO: `/v3/projects/6aad6319b28d42608af9f827ce139c76/users/d8cfb326d81d4212b124a5108e9eb725/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:45.341583 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354e6b84-c20e-4d78-95f4-e7d5940a11ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=6aad6319b28d42608af9f827ce139c76, user_id=d8cfb326d81d4212b124a5108e9eb725, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:45.344440 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354e6b84-c20e-4d78-95f4-e7d5940a11ef admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:45.346749 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 816/1646] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:45:45 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:45:45.351221 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51f8c70-ec40-45bb-99d3-d8ee6fe72683 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:45.368229 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 831/1647] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:45:45 2022] PUT /identity/v3/projects/6aad6319b28d42608af9f827ce139c76/users/d8cfb326d81d4212b124a5108e9eb725/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:45.370965 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51f8c70-ec40-45bb-99d3-d8ee6fe72683 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:45.372562 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354e6b84-c20e-4d78-95f4-e7d5940a11ef admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:45.439052 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:45.490306 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0038450-79a5-4cd5-a949-c54dbc83f7cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:45.491021 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0038450-79a5-4cd5-a949-c54dbc83f7cd admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:45.492506 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0038450-79a5-4cd5-a949-c54dbc83f7cd admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:45.492873 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0038450-79a5-4cd5-a949-c54dbc83f7cd admin admin] PATH_INFO: `/v3/projects/eb68a5cb7877444ebaa2db56830d080a/users/c9d2e670b03d42ac8a58f8376647da98/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:45.496416 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0038450-79a5-4cd5-a949-c54dbc83f7cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb68a5cb7877444ebaa2db56830d080a, user_id=c9d2e670b03d42ac8a58f8376647da98, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:45.499718 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0038450-79a5-4cd5-a949-c54dbc83f7cd admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:45.504540 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354e6b84-c20e-4d78-95f4-e7d5940a11ef admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:45.525267 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fad5da6-481c-4e97-b87c-c92a637ec2b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:45.525841 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 817/1648] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:45:45 2022] PUT /identity/v3/projects/eb68a5cb7877444ebaa2db56830d080a/users/c9d2e670b03d42ac8a58f8376647da98/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 08:45:45.526491 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fad5da6-481c-4e97-b87c-c92a637ec2b0 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:45.526957 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fad5da6-481c-4e97-b87c-c92a637ec2b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:45.527417 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fad5da6-481c-4e97-b87c-c92a637ec2b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:45.527833 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b07a27-4bc3-47ac-b176-c6d611e0d8e1 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:45.528469 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fad5da6-481c-4e97-b87c-c92a637ec2b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:45.529288 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b07a27-4bc3-47ac-b176-c6d611e0d8e1 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:45.529585 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b07a27-4bc3-47ac-b176-c6d611e0d8e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:45.530787 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fad5da6-481c-4e97-b87c-c92a637ec2b0 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:45.551655 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 832/1649] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:45:45 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:45:45.553004 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fad5da6-481c-4e97-b87c-c92a637ec2b0 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:45.585834 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fad5da6-481c-4e97-b87c-c92a637ec2b0 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:45.607035 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-94b07a27-4bc3-47ac-b176-c6d611e0d8e1 None None] MFA Rules not processed for user `c9d2e670b03d42ac8a58f8376647da98`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:45.609706 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-94b07a27-4bc3-47ac-b176-c6d611e0d8e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:45.732371 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055dc892-acdc-4509-bda4-1a328d6fbe24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:45.732880 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055dc892-acdc-4509-bda4-1a328d6fbe24 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:45.733169 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055dc892-acdc-4509-bda4-1a328d6fbe24 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:45.733448 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055dc892-acdc-4509-bda4-1a328d6fbe24 admin admin] PATH_INFO: `/v3/projects/6aad6319b28d42608af9f827ce139c76/users/d8cfb326d81d4212b124a5108e9eb725/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:45.735621 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055dc892-acdc-4509-bda4-1a328d6fbe24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6aad6319b28d42608af9f827ce139c76, user_id=d8cfb326d81d4212b124a5108e9eb725, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:45.738338 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055dc892-acdc-4509-bda4-1a328d6fbe24 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:45.751769 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 833/1650] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:45:45 2022] PUT /identity/v3/projects/6aad6319b28d42608af9f827ce139c76/users/d8cfb326d81d4212b124a5108e9eb725/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:45.752892 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055dc892-acdc-4509-bda4-1a328d6fbe24 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:45.759208 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-055dc892-acdc-4509-bda4-1a328d6fbe24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:45.770356 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 818/1651] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:45 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:45:45.772736 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd29e52-3b96-4af6-8f52-899fef04768c None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:45.773183 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd29e52-3b96-4af6-8f52-899fef04768c None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:45.773472 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd29e52-3b96-4af6-8f52-899fef04768c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:45.811314 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-7dd29e52-3b96-4af6-8f52-899fef04768c None None] MFA Rules not processed for user `d8cfb326d81d4212b124a5108e9eb725`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:45.812762 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-7dd29e52-3b96-4af6-8f52-899fef04768c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:45.824332 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:45.847418 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055dc892-acdc-4509-bda4-1a328d6fbe24 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:45.855110 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5657a06-d203-4cb8-8408-01478fd1de9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:45.856161 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5657a06-d203-4cb8-8408-01478fd1de9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:45.856664 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5657a06-d203-4cb8-8408-01478fd1de9c admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:45.857316 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5657a06-d203-4cb8-8408-01478fd1de9c admin admin] PATH_INFO: `/v3/projects/41cf643468b845b38413cc12216b11fd` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:45.859266 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5657a06-d203-4cb8-8408-01478fd1de9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=41cf643468b845b38413cc12216b11fd)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:45.861622 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5657a06-d203-4cb8-8408-01478fd1de9c admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:45.898065 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 819/1652] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:45 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:45:45.899023 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd29e52-3b96-4af6-8f52-899fef04768c None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:45.901515 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-7dd29e52-3b96-4af6-8f52-899fef04768c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:45.920906 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-c5657a06-d203-4cb8-8408-01478fd1de9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41cf643468b845b38413cc12216b11fd', 'request_id': 'req-c5657a06-d203-4cb8-8408-01478fd1de9c'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:45.927811 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-c5657a06-d203-4cb8-8408-01478fd1de9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41cf643468b845b38413cc12216b11fd', 'request_id': 'req-c5657a06-d203-4cb8-8408-01478fd1de9c'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:45.939813 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-c5657a06-d203-4cb8-8408-01478fd1de9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41cf643468b845b38413cc12216b11fd', 'request_id': 'req-c5657a06-d203-4cb8-8408-01478fd1de9c'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:45.946073 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:45.966956 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-c5657a06-d203-4cb8-8408-01478fd1de9c admin admin] The token cache is being invalidate because project 41cf643468b845b38413cc12216b11fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 08:45:45.966956 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-c5657a06-d203-4cb8-8408-01478fd1de9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:45.967553 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-c5657a06-d203-4cb8-8408-01478fd1de9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:45.969191 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 834/1653] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 08:45:45 2022] DELETE /identity/v3/projects/41cf643468b845b38413cc12216b11fd => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:45.974934 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5657a06-d203-4cb8-8408-01478fd1de9c admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:45.977262 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-c5657a06-d203-4cb8-8408-01478fd1de9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:45.981259 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd29e52-3b96-4af6-8f52-899fef04768c None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:46.040922 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571a7532-f9c2-4006-9884-7c491f7de7a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:46.041629 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a7532-f9c2-4006-9884-7c491f7de7a2 service cinder] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:46.042246 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a7532-f9c2-4006-9884-7c491f7de7a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:46.042246 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a7532-f9c2-4006-9884-7c491f7de7a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:46.044444 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-571a7532-f9c2-4006-9884-7c491f7de7a2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:46.085735 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5657a06-d203-4cb8-8408-01478fd1de9c admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:46.096332 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef9c87b-2821-411d-8598-8f6e2840a115 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:46.096332 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef9c87b-2821-411d-8598-8f6e2840a115 service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:46.096332 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef9c87b-2821-411d-8598-8f6e2840a115 service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:46.096332 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef9c87b-2821-411d-8598-8f6e2840a115 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:46.099393 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-bef9c87b-2821-411d-8598-8f6e2840a115 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:46.190183 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571a7532-f9c2-4006-9884-7c491f7de7a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:46.192205 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571a7532-f9c2-4006-9884-7c491f7de7a2 service cinder] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:46.204856 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 820/1654] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:45:45 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:45:46.206330 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938943be-927f-47a5-8915-45a7cde0c779 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:46.206824 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938943be-927f-47a5-8915-45a7cde0c779 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:46.206943 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938943be-927f-47a5-8915-45a7cde0c779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:46.240871 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-938943be-927f-47a5-8915-45a7cde0c779 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:46.242691 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-938943be-927f-47a5-8915-45a7cde0c779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:46.254238 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef9c87b-2821-411d-8598-8f6e2840a115 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:46.255715 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef9c87b-2821-411d-8598-8f6e2840a115 service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:46.294055 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 835/1655] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:45:45 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:45:46.410228 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 821/1656] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:45:46.420629 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84dc88c-4f1e-4e35-9a8c-02e1082c94e5 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:46.420921 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84dc88c-4f1e-4e35-9a8c-02e1082c94e5 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:46.421204 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84dc88c-4f1e-4e35-9a8c-02e1082c94e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:46.482926 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-c84dc88c-4f1e-4e35-9a8c-02e1082c94e5 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:46.484536 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-c84dc88c-4f1e-4e35-9a8c-02e1082c94e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:46.512780 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938943be-927f-47a5-8915-45a7cde0c779 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:46.514784 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-938943be-927f-47a5-8915-45a7cde0c779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:46.515295 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 836/1657] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:45:46.516205 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84dc88c-4f1e-4e35-9a8c-02e1082c94e5 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:46.529332 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84dc88c-4f1e-4e35-9a8c-02e1082c94e5 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:46.535193 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a44340-ab85-483c-8d8e-de2dd7f03a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:46.536760 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a44340-ab85-483c-8d8e-de2dd7f03a8d admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:46.537085 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a44340-ab85-483c-8d8e-de2dd7f03a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:46.537314 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a44340-ab85-483c-8d8e-de2dd7f03a8d admin admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:46.539276 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a44340-ab85-483c-8d8e-de2dd7f03a8d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:45:46.539590 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a44340-ab85-483c-8d8e-de2dd7f03a8d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:46.540896 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938943be-927f-47a5-8915-45a7cde0c779 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:46.543097 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a44340-ab85-483c-8d8e-de2dd7f03a8d admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:46.550737 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97b545e-a83f-4a21-b4d0-cbb0bf43b78f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:46.553409 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97b545e-a83f-4a21-b4d0-cbb0bf43b78f service cinder] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:46.556920 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97b545e-a83f-4a21-b4d0-cbb0bf43b78f service cinder] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:46.557992 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97b545e-a83f-4a21-b4d0-cbb0bf43b78f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:46.561816 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-e97b545e-a83f-4a21-b4d0-cbb0bf43b78f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:46.583220 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 837/1658] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 08:45:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:45:46.584401 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a44340-ab85-483c-8d8e-de2dd7f03a8d admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:46.594607 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:46.604933 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a44340-ab85-483c-8d8e-de2dd7f03a8d admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:46.609630 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:46.623893 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8983b3b-74c0-4b9a-9f64-09adc07be769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:46.624531 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8983b3b-74c0-4b9a-9f64-09adc07be769 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:46.638043 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8983b3b-74c0-4b9a-9f64-09adc07be769 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:46.638343 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8983b3b-74c0-4b9a-9f64-09adc07be769 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:46.639102 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8983b3b-74c0-4b9a-9f64-09adc07be769 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:46.641293 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97b545e-a83f-4a21-b4d0-cbb0bf43b78f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:46.641933 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8983b3b-74c0-4b9a-9f64-09adc07be769 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:46.643379 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97b545e-a83f-4a21-b4d0-cbb0bf43b78f service cinder] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:46.674373 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 822/1659] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:45:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 08:45:46.701966 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 838/1660] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 08:45:46 2022] POST /identity/v3/projects => generated 375 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:45:46.703259 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97b545e-a83f-4a21-b4d0-cbb0bf43b78f service cinder] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:46.723912 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97b545e-a83f-4a21-b4d0-cbb0bf43b78f service cinder] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:46.817742 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01278b3c-87f4-4229-8a0a-b1c279e214f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:46.818596 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01278b3c-87f4-4229-8a0a-b1c279e214f3 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:46.818950 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01278b3c-87f4-4229-8a0a-b1c279e214f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:46.820013 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01278b3c-87f4-4229-8a0a-b1c279e214f3 admin admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:46.820880 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01278b3c-87f4-4229-8a0a-b1c279e214f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:46.823096 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01278b3c-87f4-4229-8a0a-b1c279e214f3 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:46.853743 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 823/1661] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:45:46 2022] POST /identity/v3/users => generated 288 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:45:46.857413 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8983b3b-74c0-4b9a-9f64-09adc07be769 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:46.870248 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8983b3b-74c0-4b9a-9f64-09adc07be769 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:46.880692 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad8b8fc-ffc5-47fc-8be9-f98a2a56a5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:46.881495 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad8b8fc-ffc5-47fc-8be9-f98a2a56a5e6 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:46.882552 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad8b8fc-ffc5-47fc-8be9-f98a2a56a5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:46.882967 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad8b8fc-ffc5-47fc-8be9-f98a2a56a5e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:46.883790 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad8b8fc-ffc5-47fc-8be9-f98a2a56a5e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:46.885532 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad8b8fc-ffc5-47fc-8be9-f98a2a56a5e6 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:46.903685 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 839/1662] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:45:46 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:45:46.907719 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01278b3c-87f4-4229-8a0a-b1c279e214f3 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:46.919518 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01278b3c-87f4-4229-8a0a-b1c279e214f3 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:46.925583 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5fdd43-71c8-4b0d-bbb1-ca7e88c1f0b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:46.926482 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5fdd43-71c8-4b0d-bbb1-ca7e88c1f0b8 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:46.926937 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5fdd43-71c8-4b0d-bbb1-ca7e88c1f0b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:46.927215 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5fdd43-71c8-4b0d-bbb1-ca7e88c1f0b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:46.927879 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5fdd43-71c8-4b0d-bbb1-ca7e88c1f0b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:46.930099 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5fdd43-71c8-4b0d-bbb1-ca7e88c1f0b8 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:46.948325 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 824/1663] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:45:46 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:45:46.953281 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad8b8fc-ffc5-47fc-8be9-f98a2a56a5e6 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:46.967387 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad8b8fc-ffc5-47fc-8be9-f98a2a56a5e6 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:46.973632 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4596636b-e2ef-46e9-84dd-fd17d94dc45a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:46.975503 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4596636b-e2ef-46e9-84dd-fd17d94dc45a admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:46.975986 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4596636b-e2ef-46e9-84dd-fd17d94dc45a admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:46.976639 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4596636b-e2ef-46e9-84dd-fd17d94dc45a admin admin] PATH_INFO: `/v3/projects/80799151ef6c4be8b1b436daebe8a98b/users/bf66bc4c78b94c51b21097e460fa45d8/roles/3248f0268738489b883041851d681bb7` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:47.019001 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4596636b-e2ef-46e9-84dd-fd17d94dc45a admin admin] RBAC: Authorizing `identity:create_grant(project_id=80799151ef6c4be8b1b436daebe8a98b, user_id=bf66bc4c78b94c51b21097e460fa45d8, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:47.023513 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4596636b-e2ef-46e9-84dd-fd17d94dc45a admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:47.051760 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 840/1664] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:45:46 2022] PUT /identity/v3/projects/80799151ef6c4be8b1b436daebe8a98b/users/bf66bc4c78b94c51b21097e460fa45d8/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:47.062533 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d127e2-6131-4d24-a461-dd59f80f62fc None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:47.062890 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d127e2-6131-4d24-a461-dd59f80f62fc None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:47.063109 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d127e2-6131-4d24-a461-dd59f80f62fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:47.094031 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4596636b-e2ef-46e9-84dd-fd17d94dc45a admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:47.098097 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-4596636b-e2ef-46e9-84dd-fd17d94dc45a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:47.136038 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-64d127e2-6131-4d24-a461-dd59f80f62fc None None] MFA Rules not processed for user `bf66bc4c78b94c51b21097e460fa45d8`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:47.137641 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-64d127e2-6131-4d24-a461-dd59f80f62fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:47.230407 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4596636b-e2ef-46e9-84dd-fd17d94dc45a admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:47.236513 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addef113-7c0c-43b5-b2c1-a2e9c2c3f315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:47.237658 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addef113-7c0c-43b5-b2c1-a2e9c2c3f315 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:47.238138 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addef113-7c0c-43b5-b2c1-a2e9c2c3f315 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:47.238674 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addef113-7c0c-43b5-b2c1-a2e9c2c3f315 admin admin] PATH_INFO: `/v3/users/3ec0aebc837d488f84806c266dbaf392` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:47.240395 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addef113-7c0c-43b5-b2c1-a2e9c2c3f315 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ec0aebc837d488f84806c266dbaf392)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:47.243854 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addef113-7c0c-43b5-b2c1-a2e9c2c3f315 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:47.319264 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-addef113-7c0c-43b5-b2c1-a2e9c2c3f315 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ec0aebc837d488f84806c266dbaf392'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:47.320145 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-addef113-7c0c-43b5-b2c1-a2e9c2c3f315 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ec0aebc837d488f84806c266dbaf392'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:47.332258 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-addef113-7c0c-43b5-b2c1-a2e9c2c3f315 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ec0aebc837d488f84806c266dbaf392'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:47.352849 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-addef113-7c0c-43b5-b2c1-a2e9c2c3f315 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ec0aebc837d488f84806c266dbaf392'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:47.363191 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-addef113-7c0c-43b5-b2c1-a2e9c2c3f315 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ec0aebc837d488f84806c266dbaf392'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:47.372360 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 841/1665] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 08:45:47 2022] DELETE /identity/v3/users/3ec0aebc837d488f84806c266dbaf392 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 08:45:47.373407 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addef113-7c0c-43b5-b2c1-a2e9c2c3f315 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:47.375280 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-addef113-7c0c-43b5-b2c1-a2e9c2c3f315 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:47.416151 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 825/1666] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:45:47.417792 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa305b2c-5f35-4c57-849e-05ad80b1eb64 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:47.418239 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa305b2c-5f35-4c57-849e-05ad80b1eb64 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:47.418700 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa305b2c-5f35-4c57-849e-05ad80b1eb64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:47.439435 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:47.442962 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-aa305b2c-5f35-4c57-849e-05ad80b1eb64 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:47.444859 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-aa305b2c-5f35-4c57-849e-05ad80b1eb64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:47.500040 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:47.517838 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addef113-7c0c-43b5-b2c1-a2e9c2c3f315 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:47.528641 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1315f016-0bf2-4598-a686-71681fa2389a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:47.528641 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1315f016-0bf2-4598-a686-71681fa2389a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:47.528821 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1315f016-0bf2-4598-a686-71681fa2389a admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:47.529070 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1315f016-0bf2-4598-a686-71681fa2389a admin admin] PATH_INFO: `/v3/users/d1d735a17e994291ab7e03c67ca139e0` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:47.530564 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1315f016-0bf2-4598-a686-71681fa2389a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1d735a17e994291ab7e03c67ca139e0)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:47.532700 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1315f016-0bf2-4598-a686-71681fa2389a admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:47.533287 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 826/1667] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:45:47.535307 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa305b2c-5f35-4c57-849e-05ad80b1eb64 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:47.541182 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-aa305b2c-5f35-4c57-849e-05ad80b1eb64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:47.604654 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-1315f016-0bf2-4598-a686-71681fa2389a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1d735a17e994291ab7e03c67ca139e0'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:47.605800 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-1315f016-0bf2-4598-a686-71681fa2389a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1d735a17e994291ab7e03c67ca139e0'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:47.616119 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa305b2c-5f35-4c57-849e-05ad80b1eb64 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:47.619763 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3db4b8-3090-4b88-bee3-e2d91cbe40e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:47.620380 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3db4b8-3090-4b88-bee3-e2d91cbe40e5 service cinder] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:47.620679 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3db4b8-3090-4b88-bee3-e2d91cbe40e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:47.620950 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3db4b8-3090-4b88-bee3-e2d91cbe40e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:47.623007 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-5b3db4b8-3090-4b88-bee3-e2d91cbe40e5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:47.624523 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-1315f016-0bf2-4598-a686-71681fa2389a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1d735a17e994291ab7e03c67ca139e0'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:47.644330 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-1315f016-0bf2-4598-a686-71681fa2389a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1d735a17e994291ab7e03c67ca139e0'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:47.654283 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-1315f016-0bf2-4598-a686-71681fa2389a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1d735a17e994291ab7e03c67ca139e0'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:47.666579 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 842/1668] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 08:45:47 2022] DELETE /identity/v3/users/d1d735a17e994291ab7e03c67ca139e0 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:47.666579 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1315f016-0bf2-4598-a686-71681fa2389a admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:47.666784 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-1315f016-0bf2-4598-a686-71681fa2389a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:47.705421 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3db4b8-3090-4b88-bee3-e2d91cbe40e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:47.706920 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3db4b8-3090-4b88-bee3-e2d91cbe40e5 service cinder] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:47.718400 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 827/1669] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:45:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:45:47.719732 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f2fef-45fe-444d-bf28-2bca46342532 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:47.720054 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f2fef-45fe-444d-bf28-2bca46342532 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:47.720325 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f2fef-45fe-444d-bf28-2bca46342532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:47.741361 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-f96f2fef-45fe-444d-bf28-2bca46342532 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:47.742711 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-f96f2fef-45fe-444d-bf28-2bca46342532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:47.751571 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1315f016-0bf2-4598-a686-71681fa2389a admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:47.757806 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb785e7-ba83-416e-87e8-3aeee0239d39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:47.758650 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb785e7-ba83-416e-87e8-3aeee0239d39 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:47.759010 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb785e7-ba83-416e-87e8-3aeee0239d39 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:47.759357 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb785e7-ba83-416e-87e8-3aeee0239d39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:47.761316 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-7cb785e7-ba83-416e-87e8-3aeee0239d39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:47.841921 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 828/1670] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:45:47.848389 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96f2fef-45fe-444d-bf28-2bca46342532 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:47.869542 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:47.872675 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb785e7-ba83-416e-87e8-3aeee0239d39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:47.874726 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96f2fef-45fe-444d-bf28-2bca46342532 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:47.874945 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb785e7-ba83-416e-87e8-3aeee0239d39 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:47.881228 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941c567a-c17e-44bf-94ae-a01345974740 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:47.882553 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c567a-c17e-44bf-94ae-a01345974740 service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:47.882914 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c567a-c17e-44bf-94ae-a01345974740 service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:47.883443 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c567a-c17e-44bf-94ae-a01345974740 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:47.899264 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941c567a-c17e-44bf-94ae-a01345974740 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:47.900538 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 843/1671] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:45:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:45:47.912794 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941c567a-c17e-44bf-94ae-a01345974740 service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:47.935579 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 829/1672] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:45:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 08:45:48.304022 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb785e7-ba83-416e-87e8-3aeee0239d39 service neutron] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:48.306919 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-7cb785e7-ba83-416e-87e8-3aeee0239d39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:48.332549 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb785e7-ba83-416e-87e8-3aeee0239d39 service neutron] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:48.340168 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f15ae4-7551-4165-b352-e4d24dd2038c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:48.340804 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f15ae4-7551-4165-b352-e4d24dd2038c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:48.341195 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f15ae4-7551-4165-b352-e4d24dd2038c admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:48.341505 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f15ae4-7551-4165-b352-e4d24dd2038c admin admin] PATH_INFO: `/v3/projects/33a0fa812dc14fa2be10b911a914b44f` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:48.342664 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f15ae4-7551-4165-b352-e4d24dd2038c admin admin] RBAC: Authorizing `identity:delete_project(project_id=33a0fa812dc14fa2be10b911a914b44f)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:48.344921 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f15ae4-7551-4165-b352-e4d24dd2038c admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:48.386077 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941c567a-c17e-44bf-94ae-a01345974740 service neutron] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:48.387962 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-941c567a-c17e-44bf-94ae-a01345974740 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:48.410289 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941c567a-c17e-44bf-94ae-a01345974740 service neutron] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:48.426188 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-54f15ae4-7551-4165-b352-e4d24dd2038c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33a0fa812dc14fa2be10b911a914b44f', 'request_id': 'req-54f15ae4-7551-4165-b352-e4d24dd2038c'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:48.433043 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-54f15ae4-7551-4165-b352-e4d24dd2038c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33a0fa812dc14fa2be10b911a914b44f', 'request_id': 'req-54f15ae4-7551-4165-b352-e4d24dd2038c'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:48.437791 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:48.441666 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-54f15ae4-7551-4165-b352-e4d24dd2038c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33a0fa812dc14fa2be10b911a914b44f', 'request_id': 'req-54f15ae4-7551-4165-b352-e4d24dd2038c'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:48.452876 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-54f15ae4-7551-4165-b352-e4d24dd2038c admin admin] The token cache is being invalidate because project 33a0fa812dc14fa2be10b911a914b44f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 08:45:48.453171 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-54f15ae4-7551-4165-b352-e4d24dd2038c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:48.453843 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-54f15ae4-7551-4165-b352-e4d24dd2038c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:48.455345 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 844/1673] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 08:45:48 2022] DELETE /identity/v3/projects/33a0fa812dc14fa2be10b911a914b44f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:48.459246 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f15ae4-7551-4165-b352-e4d24dd2038c admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:48.461160 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-54f15ae4-7551-4165-b352-e4d24dd2038c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:48.470497 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb70215-b3b4-4f57-b23c-3e5ad47c7416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:48.471200 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb70215-b3b4-4f57-b23c-3e5ad47c7416 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:48.471483 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb70215-b3b4-4f57-b23c-3e5ad47c7416 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:48.471758 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb70215-b3b4-4f57-b23c-3e5ad47c7416 admin admin] PATH_INFO: `/v3/projects/5347240119704ed583006c50eebc0c7c` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:48.472855 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb70215-b3b4-4f57-b23c-3e5ad47c7416 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5347240119704ed583006c50eebc0c7c)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:48.474842 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb70215-b3b4-4f57-b23c-3e5ad47c7416 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:48.527986 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-ceb70215-b3b4-4f57-b23c-3e5ad47c7416 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5347240119704ed583006c50eebc0c7c', 'request_id': 'req-ceb70215-b3b4-4f57-b23c-3e5ad47c7416'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:48.533099 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f15ae4-7551-4165-b352-e4d24dd2038c admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:48.534400 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-ceb70215-b3b4-4f57-b23c-3e5ad47c7416 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5347240119704ed583006c50eebc0c7c', 'request_id': 'req-ceb70215-b3b4-4f57-b23c-3e5ad47c7416'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:48.545047 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-ceb70215-b3b4-4f57-b23c-3e5ad47c7416 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5347240119704ed583006c50eebc0c7c', 'request_id': 'req-ceb70215-b3b4-4f57-b23c-3e5ad47c7416'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:48.560057 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-ceb70215-b3b4-4f57-b23c-3e5ad47c7416 admin admin] The token cache is being invalidate because project 5347240119704ed583006c50eebc0c7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 08:45:48.560335 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-ceb70215-b3b4-4f57-b23c-3e5ad47c7416 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:48.560949 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-ceb70215-b3b4-4f57-b23c-3e5ad47c7416 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:48.563316 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 830/1674] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 08:45:48 2022] DELETE /identity/v3/projects/5347240119704ed583006c50eebc0c7c => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:48.566887 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb70215-b3b4-4f57-b23c-3e5ad47c7416 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:48.568221 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-ceb70215-b3b4-4f57-b23c-3e5ad47c7416 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:48.607054 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d35654e-11e6-46fd-8ff6-7ca011b72714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:48.607746 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d35654e-11e6-46fd-8ff6-7ca011b72714 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:48.608024 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d35654e-11e6-46fd-8ff6-7ca011b72714 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:48.609122 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d35654e-11e6-46fd-8ff6-7ca011b72714 admin admin] PATH_INFO: `/v3/users/d8cfb326d81d4212b124a5108e9eb725` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:48.609852 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d35654e-11e6-46fd-8ff6-7ca011b72714 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8cfb326d81d4212b124a5108e9eb725)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:48.611841 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d35654e-11e6-46fd-8ff6-7ca011b72714 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:48.641960 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb70215-b3b4-4f57-b23c-3e5ad47c7416 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:48.647127 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575c7c42-699f-438c-8006-f47bb28b5a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:48.648497 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c7c42-699f-438c-8006-f47bb28b5a18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:48.648846 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c7c42-699f-438c-8006-f47bb28b5a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:48.649292 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c7c42-699f-438c-8006-f47bb28b5a18 admin admin] PATH_INFO: `/v3/users/c9d2e670b03d42ac8a58f8376647da98` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:48.650059 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:48.650692 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575c7c42-699f-438c-8006-f47bb28b5a18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9d2e670b03d42ac8a58f8376647da98)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:48.652517 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575c7c42-699f-438c-8006-f47bb28b5a18 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:48.681703 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-1d35654e-11e6-46fd-8ff6-7ca011b72714 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8cfb326d81d4212b124a5108e9eb725'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:48.681703 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-1d35654e-11e6-46fd-8ff6-7ca011b72714 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8cfb326d81d4212b124a5108e9eb725'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:48.686215 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-1d35654e-11e6-46fd-8ff6-7ca011b72714 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8cfb326d81d4212b124a5108e9eb725'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:48.700123 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-1d35654e-11e6-46fd-8ff6-7ca011b72714 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8cfb326d81d4212b124a5108e9eb725'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:48.707770 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-575c7c42-699f-438c-8006-f47bb28b5a18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9d2e670b03d42ac8a58f8376647da98'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:48.707920 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-1d35654e-11e6-46fd-8ff6-7ca011b72714 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8cfb326d81d4212b124a5108e9eb725'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:48.708535 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-575c7c42-699f-438c-8006-f47bb28b5a18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9d2e670b03d42ac8a58f8376647da98'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:48.715989 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 845/1675] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 08:45:48 2022] DELETE /identity/v3/users/d8cfb326d81d4212b124a5108e9eb725 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 08:45:48.717658 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:48.725999 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-575c7c42-699f-438c-8006-f47bb28b5a18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9d2e670b03d42ac8a58f8376647da98'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:48.734026 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-575c7c42-699f-438c-8006-f47bb28b5a18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9d2e670b03d42ac8a58f8376647da98'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:48.734701 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-575c7c42-699f-438c-8006-f47bb28b5a18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9d2e670b03d42ac8a58f8376647da98'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:48.747644 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 831/1676] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 08:45:48 2022] DELETE /identity/v3/users/c9d2e670b03d42ac8a58f8376647da98 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 08:45:48.990356 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d35654e-11e6-46fd-8ff6-7ca011b72714 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:48.992330 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-1d35654e-11e6-46fd-8ff6-7ca011b72714 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:49.031092 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575c7c42-699f-438c-8006-f47bb28b5a18 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:49.032571 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-575c7c42-699f-438c-8006-f47bb28b5a18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:49.096977 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d35654e-11e6-46fd-8ff6-7ca011b72714 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:49.105281 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b425816e-a319-475c-9b9e-a92b725d972b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:49.106101 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b425816e-a319-475c-9b9e-a92b725d972b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:49.106685 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b425816e-a319-475c-9b9e-a92b725d972b admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:49.106978 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b425816e-a319-475c-9b9e-a92b725d972b admin admin] PATH_INFO: `/v3/projects/6aad6319b28d42608af9f827ce139c76` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:49.108758 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575c7c42-699f-438c-8006-f47bb28b5a18 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:49.108866 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b425816e-a319-475c-9b9e-a92b725d972b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6aad6319b28d42608af9f827ce139c76)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:49.111018 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b425816e-a319-475c-9b9e-a92b725d972b admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:49.114438 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55db77f-d12f-4394-b64f-168c1bfcacd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:49.115254 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55db77f-d12f-4394-b64f-168c1bfcacd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:49.115662 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55db77f-d12f-4394-b64f-168c1bfcacd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:49.116541 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55db77f-d12f-4394-b64f-168c1bfcacd4 admin admin] PATH_INFO: `/v3/projects/eb68a5cb7877444ebaa2db56830d080a` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:49.117899 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55db77f-d12f-4394-b64f-168c1bfcacd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb68a5cb7877444ebaa2db56830d080a)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:49.119998 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55db77f-d12f-4394-b64f-168c1bfcacd4 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:49.170750 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b425816e-a319-475c-9b9e-a92b725d972b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6aad6319b28d42608af9f827ce139c76', 'request_id': 'req-b425816e-a319-475c-9b9e-a92b725d972b'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:49.177470 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b425816e-a319-475c-9b9e-a92b725d972b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6aad6319b28d42608af9f827ce139c76', 'request_id': 'req-b425816e-a319-475c-9b9e-a92b725d972b'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:49.181773 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-d55db77f-d12f-4394-b64f-168c1bfcacd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb68a5cb7877444ebaa2db56830d080a', 'request_id': 'req-d55db77f-d12f-4394-b64f-168c1bfcacd4'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:49.187509 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b425816e-a319-475c-9b9e-a92b725d972b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6aad6319b28d42608af9f827ce139c76', 'request_id': 'req-b425816e-a319-475c-9b9e-a92b725d972b'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:49.189274 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-d55db77f-d12f-4394-b64f-168c1bfcacd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb68a5cb7877444ebaa2db56830d080a', 'request_id': 'req-d55db77f-d12f-4394-b64f-168c1bfcacd4'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:49.199720 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-d55db77f-d12f-4394-b64f-168c1bfcacd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb68a5cb7877444ebaa2db56830d080a', 'request_id': 'req-d55db77f-d12f-4394-b64f-168c1bfcacd4'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:49.200900 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b425816e-a319-475c-9b9e-a92b725d972b admin admin] The token cache is being invalidate because project 6aad6319b28d42608af9f827ce139c76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 08:45:49.201543 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b425816e-a319-475c-9b9e-a92b725d972b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:49.202252 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b425816e-a319-475c-9b9e-a92b725d972b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:49.203905 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 846/1677] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 08:45:48 2022] DELETE /identity/v3/projects/6aad6319b28d42608af9f827ce139c76 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:49.208921 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a940c1-6132-41f8-99de-efd21319aeb3 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:49.209198 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a940c1-6132-41f8-99de-efd21319aeb3 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:49.209470 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a940c1-6132-41f8-99de-efd21319aeb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:49.215083 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-d55db77f-d12f-4394-b64f-168c1bfcacd4 admin admin] The token cache is being invalidate because project eb68a5cb7877444ebaa2db56830d080a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 08:45:49.215378 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-d55db77f-d12f-4394-b64f-168c1bfcacd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:49.216020 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-d55db77f-d12f-4394-b64f-168c1bfcacd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:49.217509 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 832/1678] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 08:45:49 2022] DELETE /identity/v3/projects/eb68a5cb7877444ebaa2db56830d080a => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 08:45:49.224357 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1615d4-70da-4f0b-8ded-b1b7ea32f073 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:49.224641 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1615d4-70da-4f0b-8ded-b1b7ea32f073 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:49.224918 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1615d4-70da-4f0b-8ded-b1b7ea32f073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:49.234382 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-68a940c1-6132-41f8-99de-efd21319aeb3 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:49.235579 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-68a940c1-6132-41f8-99de-efd21319aeb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:49.244273 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-2a1615d4-70da-4f0b-8ded-b1b7ea32f073 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:49.245446 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-2a1615d4-70da-4f0b-8ded-b1b7ea32f073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:49.267973 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:49.301722 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 847/1679] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:45:49.306948 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 833/1680] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 08:45:49.309577 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e42daf8-bc86-4e14-9f82-36acec2938c0 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:49.310069 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e42daf8-bc86-4e14-9f82-36acec2938c0 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:49.310230 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e42daf8-bc86-4e14-9f82-36acec2938c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:49.315263 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791b1567-e596-4ec8-aa67-f72b3ce6be29 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:49.315547 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791b1567-e596-4ec8-aa67-f72b3ce6be29 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:49.315818 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791b1567-e596-4ec8-aa67-f72b3ce6be29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:49.330560 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-8e42daf8-bc86-4e14-9f82-36acec2938c0 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:49.331769 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-8e42daf8-bc86-4e14-9f82-36acec2938c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:49.333255 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:49.340168 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 848/1681] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 08:45:49.345305 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e42daf8-bc86-4e14-9f82-36acec2938c0 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:49.345548 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-791b1567-e596-4ec8-aa67-f72b3ce6be29 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:49.346908 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-791b1567-e596-4ec8-aa67-f72b3ce6be29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:49.356645 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e42daf8-bc86-4e14-9f82-36acec2938c0 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:49.358721 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 834/1682] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 08:45:49.362620 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791b1567-e596-4ec8-aa67-f72b3ce6be29 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:49.364778 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3f3ed5-7da9-478b-b3dc-c17f6420956c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:49.365396 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f3ed5-7da9-478b-b3dc-c17f6420956c admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:49.365704 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f3ed5-7da9-478b-b3dc-c17f6420956c admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:49.366592 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f3ed5-7da9-478b-b3dc-c17f6420956c admin admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:49.367260 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3f3ed5-7da9-478b-b3dc-c17f6420956c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:45:49.367695 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3f3ed5-7da9-478b-b3dc-c17f6420956c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:49.369303 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3f3ed5-7da9-478b-b3dc-c17f6420956c admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:49.372675 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791b1567-e596-4ec8-aa67-f72b3ce6be29 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:49.377369 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4123b0-0083-4c70-a034-da34f9a2b97d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:49.378078 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4123b0-0083-4c70-a034-da34f9a2b97d admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:49.378418 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4123b0-0083-4c70-a034-da34f9a2b97d admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:49.378757 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4123b0-0083-4c70-a034-da34f9a2b97d admin admin] PATH_INFO: `/v3/domains` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:49.379873 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4123b0-0083-4c70-a034-da34f9a2b97d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:45:49.380242 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4123b0-0083-4c70-a034-da34f9a2b97d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:49.387714 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 849/1683] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 08:45:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:45:49.387864 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4123b0-0083-4c70-a034-da34f9a2b97d admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:49.388504 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3f3ed5-7da9-478b-b3dc-c17f6420956c admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:49.407623 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3f3ed5-7da9-478b-b3dc-c17f6420956c admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:49.413524 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad8f12f-98f3-4d72-a9d2-c63b45af0589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:49.414238 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad8f12f-98f3-4d72-a9d2-c63b45af0589 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:49.414521 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad8f12f-98f3-4d72-a9d2-c63b45af0589 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:49.414795 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad8f12f-98f3-4d72-a9d2-c63b45af0589 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:49.418521 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 835/1684] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 08:45:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:45:49.418819 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad8f12f-98f3-4d72-a9d2-c63b45af0589 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:49.418819 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad8f12f-98f3-4d72-a9d2-c63b45af0589 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:49.419449 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4123b0-0083-4c70-a034-da34f9a2b97d admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:49.430943 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4123b0-0083-4c70-a034-da34f9a2b97d admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:49.435761 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c5152e-17ba-4995-b055-2d15e9bc0813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:49.436835 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c5152e-17ba-4995-b055-2d15e9bc0813 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:49.437121 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c5152e-17ba-4995-b055-2d15e9bc0813 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:49.437425 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c5152e-17ba-4995-b055-2d15e9bc0813 admin admin] PATH_INFO: `/v3/projects` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:49.438071 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c5152e-17ba-4995-b055-2d15e9bc0813 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:49.439965 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c5152e-17ba-4995-b055-2d15e9bc0813 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:49.452767 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 850/1685] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 08:45:49 2022] POST /identity/v3/projects => generated 385 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:45:49.453909 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad8f12f-98f3-4d72-a9d2-c63b45af0589 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:49.477879 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 836/1686] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 08:45:49 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:45:49.480187 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c5152e-17ba-4995-b055-2d15e9bc0813 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:49.522587 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad8f12f-98f3-4d72-a9d2-c63b45af0589 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:49.558752 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c5152e-17ba-4995-b055-2d15e9bc0813 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:49.563651 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6421a26-b439-4fc9-b8ee-64ddd28dcd3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:49.564262 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6421a26-b439-4fc9-b8ee-64ddd28dcd3d admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:49.564542 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6421a26-b439-4fc9-b8ee-64ddd28dcd3d admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:49.564817 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6421a26-b439-4fc9-b8ee-64ddd28dcd3d admin admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:49.565452 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6421a26-b439-4fc9-b8ee-64ddd28dcd3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:49.567430 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6421a26-b439-4fc9-b8ee-64ddd28dcd3d admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:49.590863 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab6166a-9448-4e68-b493-79d04e9f42ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:49.591476 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6166a-9448-4e68-b493-79d04e9f42ad admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:49.591756 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6166a-9448-4e68-b493-79d04e9f42ad admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:49.592153 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6166a-9448-4e68-b493-79d04e9f42ad admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:49.592289 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 837/1687] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:45:49 2022] POST /identity/v3/users => generated 287 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:45:49.592874 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab6166a-9448-4e68-b493-79d04e9f42ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:49.594978 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab6166a-9448-4e68-b493-79d04e9f42ad admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:49.596035 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6421a26-b439-4fc9-b8ee-64ddd28dcd3d admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:49.616549 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6421a26-b439-4fc9-b8ee-64ddd28dcd3d admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:49.621985 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400f6944-5fb7-4acc-ba20-2d8f97efffe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:49.622598 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f6944-5fb7-4acc-ba20-2d8f97efffe3 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:49.622885 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f6944-5fb7-4acc-ba20-2d8f97efffe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:49.623280 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f6944-5fb7-4acc-ba20-2d8f97efffe3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:49.623869 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400f6944-5fb7-4acc-ba20-2d8f97efffe3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:49.624381 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 851/1688] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:45:49 2022] POST /identity/v3/users => generated 293 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:45:49.625320 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab6166a-9448-4e68-b493-79d04e9f42ad admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:49.625563 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400f6944-5fb7-4acc-ba20-2d8f97efffe3 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:49.636883 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab6166a-9448-4e68-b493-79d04e9f42ad admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:49.640474 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 838/1689] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:45:49 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 08:45:49.644727 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400f6944-5fb7-4acc-ba20-2d8f97efffe3 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:49.657985 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19d3b9f-770d-42fc-9bcd-a24b89225326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:49.657985 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d3b9f-770d-42fc-9bcd-a24b89225326 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:49.657985 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d3b9f-770d-42fc-9bcd-a24b89225326 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:49.657985 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d3b9f-770d-42fc-9bcd-a24b89225326 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:49.658306 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19d3b9f-770d-42fc-9bcd-a24b89225326 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:49.658384 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400f6944-5fb7-4acc-ba20-2d8f97efffe3 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:49.658507 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19d3b9f-770d-42fc-9bcd-a24b89225326 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:49.660966 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a80cfb-73ca-4d58-98de-4088fd6f514f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:49.661603 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a80cfb-73ca-4d58-98de-4088fd6f514f admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:49.661938 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a80cfb-73ca-4d58-98de-4088fd6f514f admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:49.662231 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a80cfb-73ca-4d58-98de-4088fd6f514f admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:49.663413 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a80cfb-73ca-4d58-98de-4088fd6f514f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:49.665217 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a80cfb-73ca-4d58-98de-4088fd6f514f admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:49.675142 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 852/1690] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:45:49 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:45:49.676883 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 839/1691] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:45:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 08:45:49.681392 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a80cfb-73ca-4d58-98de-4088fd6f514f admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:49.681536 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19d3b9f-770d-42fc-9bcd-a24b89225326 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:49.693570 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a80cfb-73ca-4d58-98de-4088fd6f514f admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:49.694475 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19d3b9f-770d-42fc-9bcd-a24b89225326 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:49.700089 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85822e3-3548-43b7-ab94-cdc55ceee6a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:49.700696 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3450bf-e570-46ac-9855-e3e5461972c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:49.700844 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85822e3-3548-43b7-ab94-cdc55ceee6a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:49.701539 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3450bf-e570-46ac-9855-e3e5461972c1 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:49.701666 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85822e3-3548-43b7-ab94-cdc55ceee6a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:49.702041 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3450bf-e570-46ac-9855-e3e5461972c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:49.702559 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3450bf-e570-46ac-9855-e3e5461972c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:49.703512 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3450bf-e570-46ac-9855-e3e5461972c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:49.704433 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85822e3-3548-43b7-ab94-cdc55ceee6a2 admin admin] PATH_INFO: `/v3/projects/ecdc86b344484d6d88b9a5c752705224/users/07e1c6f96f0e4597bd39420b1679d7e6/roles/3248f0268738489b883041851d681bb7` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:49.705768 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3450bf-e570-46ac-9855-e3e5461972c1 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:49.723708 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85822e3-3548-43b7-ab94-cdc55ceee6a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecdc86b344484d6d88b9a5c752705224, user_id=07e1c6f96f0e4597bd39420b1679d7e6, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:49.728558 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 853/1692] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:45:49 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 08:45:49.728866 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85822e3-3548-43b7-ab94-cdc55ceee6a2 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:49.729637 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3450bf-e570-46ac-9855-e3e5461972c1 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:49.741076 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3450bf-e570-46ac-9855-e3e5461972c1 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:49.746330 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:49.747566 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 840/1693] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:45:49 2022] PUT /identity/v3/projects/ecdc86b344484d6d88b9a5c752705224/users/07e1c6f96f0e4597bd39420b1679d7e6/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:49.757044 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e082563-ace2-48e6-b6f2-a93c2a5fee19 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:49.757451 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e082563-ace2-48e6-b6f2-a93c2a5fee19 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:49.757926 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e082563-ace2-48e6-b6f2-a93c2a5fee19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:49.787736 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-6e082563-ace2-48e6-b6f2-a93c2a5fee19 None None] MFA Rules not processed for user `07e1c6f96f0e4597bd39420b1679d7e6`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:49.789603 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-6e082563-ace2-48e6-b6f2-a93c2a5fee19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:49.813862 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87076b45-1919-4cf2-a4e3-c39fc7c44483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:49.814545 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87076b45-1919-4cf2-a4e3-c39fc7c44483 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:49.814824 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87076b45-1919-4cf2-a4e3-c39fc7c44483 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:49.815102 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87076b45-1919-4cf2-a4e3-c39fc7c44483 admin admin] PATH_INFO: `/v3/projects/620a110c62394dd186f258be87fb93af/users/9dfc2409c44842979379816a14afbafa/roles/3248f0268738489b883041851d681bb7` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:49.840439 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87076b45-1919-4cf2-a4e3-c39fc7c44483 admin admin] RBAC: Authorizing `identity:create_grant(project_id=620a110c62394dd186f258be87fb93af, user_id=9dfc2409c44842979379816a14afbafa, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:49.843153 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87076b45-1919-4cf2-a4e3-c39fc7c44483 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:49.853421 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:49.863499 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 854/1694] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:45:49 2022] PUT /identity/v3/projects/620a110c62394dd186f258be87fb93af/users/9dfc2409c44842979379816a14afbafa/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:49.863897 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 841/1695] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 08:45:49.875715 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c00a890-ff0f-4355-abbe-218edd42dcbe None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:49.876149 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c00a890-ff0f-4355-abbe-218edd42dcbe None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:49.876499 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c00a890-ff0f-4355-abbe-218edd42dcbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:49.876857 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e082563-ace2-48e6-b6f2-a93c2a5fee19 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:49.878337 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-6e082563-ace2-48e6-b6f2-a93c2a5fee19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:49.908471 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-4c00a890-ff0f-4355-abbe-218edd42dcbe None None] MFA Rules not processed for user `9dfc2409c44842979379816a14afbafa`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:49.909722 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-4c00a890-ff0f-4355-abbe-218edd42dcbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:49.942290 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e082563-ace2-48e6-b6f2-a93c2a5fee19 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:49.982252 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 855/1696] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:45:49.999144 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c00a890-ff0f-4355-abbe-218edd42dcbe None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:50.000634 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-4c00a890-ff0f-4355-abbe-218edd42dcbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:50.010758 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49586b5e-935d-4b9f-ad88-349f0b6b2f45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:50.012180 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49586b5e-935d-4b9f-ad88-349f0b6b2f45 service cinder] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:50.012480 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49586b5e-935d-4b9f-ad88-349f0b6b2f45 service cinder] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:50.012791 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49586b5e-935d-4b9f-ad88-349f0b6b2f45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:50.073165 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c00a890-ff0f-4355-abbe-218edd42dcbe None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:50.079051 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0cdff3-4913-4f7f-b007-2ed5b012563c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:50.079207 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49586b5e-935d-4b9f-ad88-349f0b6b2f45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:50.079666 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0cdff3-4913-4f7f-b007-2ed5b012563c service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:50.079952 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0cdff3-4913-4f7f-b007-2ed5b012563c service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:50.080229 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0cdff3-4913-4f7f-b007-2ed5b012563c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:50.080643 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49586b5e-935d-4b9f-ad88-349f0b6b2f45 service cinder] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:50.090292 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0cdff3-4913-4f7f-b007-2ed5b012563c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:50.090851 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 842/1697] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:45:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 08:45:50.091991 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0cdff3-4913-4f7f-b007-2ed5b012563c service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:50.102454 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 856/1698] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:45:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:45:50.112950 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49586b5e-935d-4b9f-ad88-349f0b6b2f45 service cinder] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:50.130766 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0cdff3-4913-4f7f-b007-2ed5b012563c service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:50.162669 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:50.189926 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49586b5e-935d-4b9f-ad88-349f0b6b2f45 service cinder] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:50.195355 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0c1542-804c-4196-8489-dcea90082391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:50.195982 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0c1542-804c-4196-8489-dcea90082391 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:50.196262 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0c1542-804c-4196-8489-dcea90082391 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:50.196533 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0c1542-804c-4196-8489-dcea90082391 admin admin] PATH_INFO: `/v3/users/07e1c6f96f0e4597bd39420b1679d7e6` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:50.197808 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0c1542-804c-4196-8489-dcea90082391 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07e1c6f96f0e4597bd39420b1679d7e6)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:50.199904 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0c1542-804c-4196-8489-dcea90082391 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:50.203342 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0cdff3-4913-4f7f-b007-2ed5b012563c service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:50.207962 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b985d53-4a84-49b6-9f72-6347d8ed3dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:50.208624 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b985d53-4a84-49b6-9f72-6347d8ed3dcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:50.208933 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b985d53-4a84-49b6-9f72-6347d8ed3dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:50.209210 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b985d53-4a84-49b6-9f72-6347d8ed3dcd admin admin] PATH_INFO: `/v3/users/9dfc2409c44842979379816a14afbafa` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:50.210985 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b985d53-4a84-49b6-9f72-6347d8ed3dcd admin admin] RBAC: Authorizing `identity:delete_user(user_id=9dfc2409c44842979379816a14afbafa)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:50.213286 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b985d53-4a84-49b6-9f72-6347d8ed3dcd admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:50.213406 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:50.282923 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-3c0c1542-804c-4196-8489-dcea90082391 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07e1c6f96f0e4597bd39420b1679d7e6'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:50.283644 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-3c0c1542-804c-4196-8489-dcea90082391 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07e1c6f96f0e4597bd39420b1679d7e6'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:50.291728 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-5b985d53-4a84-49b6-9f72-6347d8ed3dcd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9dfc2409c44842979379816a14afbafa'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:50.292714 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-5b985d53-4a84-49b6-9f72-6347d8ed3dcd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9dfc2409c44842979379816a14afbafa'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:50.301518 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-5b985d53-4a84-49b6-9f72-6347d8ed3dcd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9dfc2409c44842979379816a14afbafa'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:50.303858 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-3c0c1542-804c-4196-8489-dcea90082391 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07e1c6f96f0e4597bd39420b1679d7e6'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:50.311761 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-3c0c1542-804c-4196-8489-dcea90082391 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07e1c6f96f0e4597bd39420b1679d7e6'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:50.312684 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-3c0c1542-804c-4196-8489-dcea90082391 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07e1c6f96f0e4597bd39420b1679d7e6'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:50.314335 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-5b985d53-4a84-49b6-9f72-6347d8ed3dcd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9dfc2409c44842979379816a14afbafa'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:50.323536 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-5b985d53-4a84-49b6-9f72-6347d8ed3dcd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9dfc2409c44842979379816a14afbafa'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:50.332627 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 843/1699] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 08:45:50 2022] DELETE /identity/v3/users/07e1c6f96f0e4597bd39420b1679d7e6 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 08:45:50.332795 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 857/1700] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 08:45:50 2022] DELETE /identity/v3/users/9dfc2409c44842979379816a14afbafa => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 08:45:50.335504 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e1547-6005-4292-9c56-526326870234 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:50.335801 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e1547-6005-4292-9c56-526326870234 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:50.336252 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e1547-6005-4292-9c56-526326870234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:50.336372 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce1ee1-7dcc-4d48-8812-ba2601b31aad None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:50.336574 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce1ee1-7dcc-4d48-8812-ba2601b31aad None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:50.336852 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce1ee1-7dcc-4d48-8812-ba2601b31aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:50.360114 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-7c9e1547-6005-4292-9c56-526326870234 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:50.361409 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-7c9e1547-6005-4292-9c56-526326870234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:50.366675 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-b6ce1ee1-7dcc-4d48-8812-ba2601b31aad None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:50.368029 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-b6ce1ee1-7dcc-4d48-8812-ba2601b31aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:50.481723 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 844/1701] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:45:50.494145 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ce1ee1-7dcc-4d48-8812-ba2601b31aad None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:50.495253 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-b6ce1ee1-7dcc-4d48-8812-ba2601b31aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:50.523047 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 858/1702] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:45:50.550807 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9e1547-6005-4292-9c56-526326870234 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:50.553080 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-7c9e1547-6005-4292-9c56-526326870234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:50.605506 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ce1ee1-7dcc-4d48-8812-ba2601b31aad None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:50.613891 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd23ceae-539f-4f8e-9d42-cb5cee364694 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:50.614613 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd23ceae-539f-4f8e-9d42-cb5cee364694 service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:50.614905 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd23ceae-539f-4f8e-9d42-cb5cee364694 service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:50.615784 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd23ceae-539f-4f8e-9d42-cb5cee364694 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:50.629005 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd23ceae-539f-4f8e-9d42-cb5cee364694 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:50.631028 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd23ceae-539f-4f8e-9d42-cb5cee364694 service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:50.646865 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 845/1703] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:45:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 08:45:50.680616 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:50.706991 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9e1547-6005-4292-9c56-526326870234 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:50.712443 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04033cc6-3c3a-4443-abd0-852307cd1668 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:50.713370 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04033cc6-3c3a-4443-abd0-852307cd1668 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:50.713654 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04033cc6-3c3a-4443-abd0-852307cd1668 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:50.714036 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04033cc6-3c3a-4443-abd0-852307cd1668 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:50.725771 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04033cc6-3c3a-4443-abd0-852307cd1668 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:50.727314 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04033cc6-3c3a-4443-abd0-852307cd1668 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:50.739155 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 859/1704] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:45:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 08:45:51.066155 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd23ceae-539f-4f8e-9d42-cb5cee364694 service neutron] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:51.068279 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-dd23ceae-539f-4f8e-9d42-cb5cee364694 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:51.088078 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd23ceae-539f-4f8e-9d42-cb5cee364694 service neutron] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:51.094219 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496c3020-81bd-48d1-8245-1333b03b58bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:51.095035 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c3020-81bd-48d1-8245-1333b03b58bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:51.095423 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c3020-81bd-48d1-8245-1333b03b58bf admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:51.095984 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c3020-81bd-48d1-8245-1333b03b58bf admin admin] PATH_INFO: `/v3/projects/ecdc86b344484d6d88b9a5c752705224` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:51.097526 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496c3020-81bd-48d1-8245-1333b03b58bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecdc86b344484d6d88b9a5c752705224)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:51.100006 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496c3020-81bd-48d1-8245-1333b03b58bf admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:51.157103 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:51.186363 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-496c3020-81bd-48d1-8245-1333b03b58bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecdc86b344484d6d88b9a5c752705224', 'request_id': 'req-496c3020-81bd-48d1-8245-1333b03b58bf'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:51.198468 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-496c3020-81bd-48d1-8245-1333b03b58bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecdc86b344484d6d88b9a5c752705224', 'request_id': 'req-496c3020-81bd-48d1-8245-1333b03b58bf'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:51.230836 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-496c3020-81bd-48d1-8245-1333b03b58bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecdc86b344484d6d88b9a5c752705224', 'request_id': 'req-496c3020-81bd-48d1-8245-1333b03b58bf'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:51.249019 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-496c3020-81bd-48d1-8245-1333b03b58bf admin admin] The token cache is being invalidate because project ecdc86b344484d6d88b9a5c752705224 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 08:45:51.249394 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-496c3020-81bd-48d1-8245-1333b03b58bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:51.250171 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-496c3020-81bd-48d1-8245-1333b03b58bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:51.253004 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 846/1705] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 08:45:51 2022] DELETE /identity/v3/projects/ecdc86b344484d6d88b9a5c752705224 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:51.258520 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca8e1d-3226-4211-a904-cc2a9470939d None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:51.258809 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca8e1d-3226-4211-a904-cc2a9470939d None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:51.259090 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca8e1d-3226-4211-a904-cc2a9470939d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:51.311334 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-d9ca8e1d-3226-4211-a904-cc2a9470939d None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:51.311334 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-d9ca8e1d-3226-4211-a904-cc2a9470939d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:51.364524 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496c3020-81bd-48d1-8245-1333b03b58bf admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:51.366179 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-496c3020-81bd-48d1-8245-1333b03b58bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:51.429718 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 860/1706] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:45:51.433657 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc078711-c225-41cf-b2ff-0bceff176b7c None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:51.434049 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc078711-c225-41cf-b2ff-0bceff176b7c None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:51.434409 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc078711-c225-41cf-b2ff-0bceff176b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:51.454213 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-dc078711-c225-41cf-b2ff-0bceff176b7c None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:51.455471 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-dc078711-c225-41cf-b2ff-0bceff176b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:51.463912 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 861/1707] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 08:45:51.467680 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496c3020-81bd-48d1-8245-1333b03b58bf admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:51.468007 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc078711-c225-41cf-b2ff-0bceff176b7c None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:51.472409 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0ddfc0-a7e8-46a3-9a62-c05c415ba12e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:51.473074 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0ddfc0-a7e8-46a3-9a62-c05c415ba12e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:51.473359 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0ddfc0-a7e8-46a3-9a62-c05c415ba12e admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:51.473671 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0ddfc0-a7e8-46a3-9a62-c05c415ba12e admin admin] PATH_INFO: `/v3/projects/620a110c62394dd186f258be87fb93af` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:51.474860 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0ddfc0-a7e8-46a3-9a62-c05c415ba12e admin admin] RBAC: Authorizing `identity:delete_project(project_id=620a110c62394dd186f258be87fb93af)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:51.477440 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0ddfc0-a7e8-46a3-9a62-c05c415ba12e admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:51.478512 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc078711-c225-41cf-b2ff-0bceff176b7c None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:51.483436 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8294fc5-8154-4789-8fc7-02e171b30edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:51.484099 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8294fc5-8154-4789-8fc7-02e171b30edf admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:51.484395 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8294fc5-8154-4789-8fc7-02e171b30edf admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:51.484674 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8294fc5-8154-4789-8fc7-02e171b30edf admin admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:51.485415 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8294fc5-8154-4789-8fc7-02e171b30edf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:45:51.485670 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8294fc5-8154-4789-8fc7-02e171b30edf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:51.487388 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8294fc5-8154-4789-8fc7-02e171b30edf admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:51.499607 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 862/1708] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 08:45:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 08:45:51.504140 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8294fc5-8154-4789-8fc7-02e171b30edf admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:51.515021 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8294fc5-8154-4789-8fc7-02e171b30edf admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:51.528628 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-da0ddfc0-a7e8-46a3-9a62-c05c415ba12e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '620a110c62394dd186f258be87fb93af', 'request_id': 'req-da0ddfc0-a7e8-46a3-9a62-c05c415ba12e'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:51.535530 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-da0ddfc0-a7e8-46a3-9a62-c05c415ba12e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '620a110c62394dd186f258be87fb93af', 'request_id': 'req-da0ddfc0-a7e8-46a3-9a62-c05c415ba12e'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:51.544600 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-da0ddfc0-a7e8-46a3-9a62-c05c415ba12e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '620a110c62394dd186f258be87fb93af', 'request_id': 'req-da0ddfc0-a7e8-46a3-9a62-c05c415ba12e'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:51.555573 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-da0ddfc0-a7e8-46a3-9a62-c05c415ba12e admin admin] The token cache is being invalidate because project 620a110c62394dd186f258be87fb93af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 08:45:51.555868 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-da0ddfc0-a7e8-46a3-9a62-c05c415ba12e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:51.557226 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-da0ddfc0-a7e8-46a3-9a62-c05c415ba12e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:51.558780 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 847/1709] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 08:45:51 2022] DELETE /identity/v3/projects/620a110c62394dd186f258be87fb93af => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 08:45:51.584686 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075d0620-1658-4cb9-9c2f-bdab5f727e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:51.585316 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d0620-1658-4cb9-9c2f-bdab5f727e33 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:51.585601 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d0620-1658-4cb9-9c2f-bdab5f727e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:51.585920 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d0620-1658-4cb9-9c2f-bdab5f727e33 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:51.586602 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075d0620-1658-4cb9-9c2f-bdab5f727e33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:51.588597 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075d0620-1658-4cb9-9c2f-bdab5f727e33 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:51.622520 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 863/1710] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 08:45:51 2022] POST /identity/v3/projects => generated 385 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:45:51.626657 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0ddfc0-a7e8-46a3-9a62-c05c415ba12e admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:51.628064 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-da0ddfc0-a7e8-46a3-9a62-c05c415ba12e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:51.641203 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0ddfc0-a7e8-46a3-9a62-c05c415ba12e admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:51.704318 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:51.742107 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dadc448-2dba-43af-97ba-fd921e172c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:51.742746 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dadc448-2dba-43af-97ba-fd921e172c02 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:51.743031 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dadc448-2dba-43af-97ba-fd921e172c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:51.743315 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dadc448-2dba-43af-97ba-fd921e172c02 admin admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:51.743976 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dadc448-2dba-43af-97ba-fd921e172c02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:51.746383 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dadc448-2dba-43af-97ba-fd921e172c02 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:51.770981 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 848/1711] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:45:51 2022] POST /identity/v3/users => generated 293 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 08:45:51.776086 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075d0620-1658-4cb9-9c2f-bdab5f727e33 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:51.787155 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075d0620-1658-4cb9-9c2f-bdab5f727e33 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:51.792645 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e096640-1b26-4cd7-9bfb-13d5f9871cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:51.793257 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e096640-1b26-4cd7-9bfb-13d5f9871cc2 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:51.793538 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e096640-1b26-4cd7-9bfb-13d5f9871cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:51.793846 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e096640-1b26-4cd7-9bfb-13d5f9871cc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:51.794486 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e096640-1b26-4cd7-9bfb-13d5f9871cc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:51.796213 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e096640-1b26-4cd7-9bfb-13d5f9871cc2 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:51.806623 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 864/1712] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:45:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 08:45:51.810530 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dadc448-2dba-43af-97ba-fd921e172c02 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:51.820389 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dadc448-2dba-43af-97ba-fd921e172c02 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:51.824946 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbefc9d-1d1d-4869-9492-96ede2c9facf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:51.825542 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbefc9d-1d1d-4869-9492-96ede2c9facf admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:51.826503 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbefc9d-1d1d-4869-9492-96ede2c9facf admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:51.826503 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbefc9d-1d1d-4869-9492-96ede2c9facf admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:51.826759 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbefc9d-1d1d-4869-9492-96ede2c9facf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:51.828357 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbefc9d-1d1d-4869-9492-96ede2c9facf admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:51.838937 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 849/1713] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:45:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:45:51.842847 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e096640-1b26-4cd7-9bfb-13d5f9871cc2 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:51.854439 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e096640-1b26-4cd7-9bfb-13d5f9871cc2 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:51.859353 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0ebf8e-5bd4-47d2-b880-60cb64bdae20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:51.859950 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ebf8e-5bd4-47d2-b880-60cb64bdae20 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:51.860230 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ebf8e-5bd4-47d2-b880-60cb64bdae20 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:51.860506 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ebf8e-5bd4-47d2-b880-60cb64bdae20 admin admin] PATH_INFO: `/v3/projects/4377987106334f2f8dcc53977ba35fd7/users/8caf111ba96d4b67bcd466eaa03c52e0/roles/3248f0268738489b883041851d681bb7` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:51.875942 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0ebf8e-5bd4-47d2-b880-60cb64bdae20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4377987106334f2f8dcc53977ba35fd7, user_id=8caf111ba96d4b67bcd466eaa03c52e0, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:51.878750 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0ebf8e-5bd4-47d2-b880-60cb64bdae20 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:51.891696 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 865/1714] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:45:51 2022] PUT /identity/v3/projects/4377987106334f2f8dcc53977ba35fd7/users/8caf111ba96d4b67bcd466eaa03c52e0/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:51.903232 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc1a4c5-cf89-4e85-978b-1698be9e987c None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:51.903518 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc1a4c5-cf89-4e85-978b-1698be9e987c None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:51.903793 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc1a4c5-cf89-4e85-978b-1698be9e987c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:51.934151 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-2bc1a4c5-cf89-4e85-978b-1698be9e987c None None] MFA Rules not processed for user `8caf111ba96d4b67bcd466eaa03c52e0`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:51.935370 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-2bc1a4c5-cf89-4e85-978b-1698be9e987c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:51.997391 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 850/1715] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:45:52.001929 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0ebf8e-5bd4-47d2-b880-60cb64bdae20 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:52.021722 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:52.065976 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0ebf8e-5bd4-47d2-b880-60cb64bdae20 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:52.071260 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f6b9cc-a3d9-4a2f-850d-9b26f64d2aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:52.071894 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f6b9cc-a3d9-4a2f-850d-9b26f64d2aa1 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:52.072180 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f6b9cc-a3d9-4a2f-850d-9b26f64d2aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:52.072460 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f6b9cc-a3d9-4a2f-850d-9b26f64d2aa1 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:52.073104 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f6b9cc-a3d9-4a2f-850d-9b26f64d2aa1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:52.075071 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f6b9cc-a3d9-4a2f-850d-9b26f64d2aa1 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:52.101051 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 866/1716] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 08:45:52 2022] POST /identity/v3/projects => generated 383 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 08:45:52.105352 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc1a4c5-cf89-4e85-978b-1698be9e987c None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:52.117324 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc1a4c5-cf89-4e85-978b-1698be9e987c None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:52.172518 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1224fe98-f66a-4e2b-8d32-6e8e18aaf40f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:52.173125 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1224fe98-f66a-4e2b-8d32-6e8e18aaf40f admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:52.173406 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1224fe98-f66a-4e2b-8d32-6e8e18aaf40f admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:52.173707 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1224fe98-f66a-4e2b-8d32-6e8e18aaf40f admin admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:52.174334 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1224fe98-f66a-4e2b-8d32-6e8e18aaf40f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:52.176251 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1224fe98-f66a-4e2b-8d32-6e8e18aaf40f admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:52.196936 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 851/1717] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:45:52 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:45:52.200897 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f6b9cc-a3d9-4a2f-850d-9b26f64d2aa1 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:52.211641 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f6b9cc-a3d9-4a2f-850d-9b26f64d2aa1 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:52.216231 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c79078c-e47f-46ed-b887-c9cf48522b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:52.216825 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c79078c-e47f-46ed-b887-c9cf48522b83 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:52.217110 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c79078c-e47f-46ed-b887-c9cf48522b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:52.217384 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c79078c-e47f-46ed-b887-c9cf48522b83 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:52.218115 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c79078c-e47f-46ed-b887-c9cf48522b83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:52.219705 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c79078c-e47f-46ed-b887-c9cf48522b83 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:52.230474 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 867/1718] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:45:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:45:52.233914 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1224fe98-f66a-4e2b-8d32-6e8e18aaf40f admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:52.243641 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1224fe98-f66a-4e2b-8d32-6e8e18aaf40f admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:52.248302 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb25a5ae-a849-449d-a2ab-1eea910ba0be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:52.248895 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25a5ae-a849-449d-a2ab-1eea910ba0be admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:52.249180 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25a5ae-a849-449d-a2ab-1eea910ba0be admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:52.249458 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25a5ae-a849-449d-a2ab-1eea910ba0be admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:52.250148 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb25a5ae-a849-449d-a2ab-1eea910ba0be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:52.251757 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb25a5ae-a849-449d-a2ab-1eea910ba0be admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:52.261940 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 852/1719] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:45:52 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:45:52.265719 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c79078c-e47f-46ed-b887-c9cf48522b83 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:52.276177 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c79078c-e47f-46ed-b887-c9cf48522b83 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:52.280716 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daccf94-4362-41e6-8d00-3a93fbde442a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:52.281352 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daccf94-4362-41e6-8d00-3a93fbde442a admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:52.281635 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daccf94-4362-41e6-8d00-3a93fbde442a admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:52.281938 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daccf94-4362-41e6-8d00-3a93fbde442a admin admin] PATH_INFO: `/v3/projects/4eece675165a483f95378350c3a5f759/users/15105cfd810945baadb52bd4cd1ec04b/roles/3248f0268738489b883041851d681bb7` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:52.296014 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daccf94-4362-41e6-8d00-3a93fbde442a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4eece675165a483f95378350c3a5f759, user_id=15105cfd810945baadb52bd4cd1ec04b, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:52.298678 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daccf94-4362-41e6-8d00-3a93fbde442a admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:52.311050 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 868/1720] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:45:52 2022] PUT /identity/v3/projects/4eece675165a483f95378350c3a5f759/users/15105cfd810945baadb52bd4cd1ec04b/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:52.322775 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1124559-3b55-421b-a02c-9eb0421023ac None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:52.322938 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1124559-3b55-421b-a02c-9eb0421023ac None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:52.323211 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1124559-3b55-421b-a02c-9eb0421023ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:52.351235 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-d1124559-3b55-421b-a02c-9eb0421023ac None None] MFA Rules not processed for user `15105cfd810945baadb52bd4cd1ec04b`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:52.352449 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-d1124559-3b55-421b-a02c-9eb0421023ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:52.375689 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:52.447191 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 853/1721] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:45:52.448401 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daccf94-4362-41e6-8d00-3a93fbde442a admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:52.522277 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daccf94-4362-41e6-8d00-3a93fbde442a admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:52.527314 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6398a08a-5e3f-46ee-9719-92a86fb4bc6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:52.527993 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398a08a-5e3f-46ee-9719-92a86fb4bc6a admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:52.528360 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398a08a-5e3f-46ee-9719-92a86fb4bc6a admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:52.528704 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398a08a-5e3f-46ee-9719-92a86fb4bc6a admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:52.529428 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6398a08a-5e3f-46ee-9719-92a86fb4bc6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:52.531429 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6398a08a-5e3f-46ee-9719-92a86fb4bc6a admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:52.562452 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 869/1722] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 08:45:52 2022] POST /identity/v3/projects => generated 381 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:45:52.566388 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1124559-3b55-421b-a02c-9eb0421023ac None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:52.578034 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1124559-3b55-421b-a02c-9eb0421023ac None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:52.635751 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14f7188-7d68-4329-a9af-7713e6440a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:52.636370 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f7188-7d68-4329-a9af-7713e6440a83 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:52.636651 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f7188-7d68-4329-a9af-7713e6440a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:52.636927 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f7188-7d68-4329-a9af-7713e6440a83 admin admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:52.637558 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14f7188-7d68-4329-a9af-7713e6440a83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:52.639507 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14f7188-7d68-4329-a9af-7713e6440a83 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:52.661938 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 854/1723] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:45:52 2022] POST /identity/v3/users => generated 297 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:45:52.665881 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6398a08a-5e3f-46ee-9719-92a86fb4bc6a admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:52.675993 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6398a08a-5e3f-46ee-9719-92a86fb4bc6a admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:52.681093 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c4a55c-3750-4ce9-a224-bc7feb93edbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:52.681720 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c4a55c-3750-4ce9-a224-bc7feb93edbd admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:52.682006 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c4a55c-3750-4ce9-a224-bc7feb93edbd admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:52.682286 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c4a55c-3750-4ce9-a224-bc7feb93edbd admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:52.682928 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c4a55c-3750-4ce9-a224-bc7feb93edbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:52.684520 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c4a55c-3750-4ce9-a224-bc7feb93edbd admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:52.705020 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 870/1724] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:45:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 08:45:52.706451 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14f7188-7d68-4329-a9af-7713e6440a83 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:52.717460 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14f7188-7d68-4329-a9af-7713e6440a83 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:52.722251 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d61555-8303-4f85-be4d-eed51f3a695f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:52.722912 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d61555-8303-4f85-be4d-eed51f3a695f admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:52.723193 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d61555-8303-4f85-be4d-eed51f3a695f admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:52.723469 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d61555-8303-4f85-be4d-eed51f3a695f admin admin] PATH_INFO: `/v3/projects/68cdf36f2d8e4f0c8f8d5a5d6c81e538/users/ac93b6c6e7be4cd185dfd824d864a1af/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:52.738429 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d61555-8303-4f85-be4d-eed51f3a695f admin admin] RBAC: Authorizing `identity:create_grant(project_id=68cdf36f2d8e4f0c8f8d5a5d6c81e538, user_id=ac93b6c6e7be4cd185dfd824d864a1af, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:52.742575 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d61555-8303-4f85-be4d-eed51f3a695f admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:52.759778 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 855/1725] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:45:52 2022] PUT /identity/v3/projects/68cdf36f2d8e4f0c8f8d5a5d6c81e538/users/ac93b6c6e7be4cd185dfd824d864a1af/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:52.763784 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c4a55c-3750-4ce9-a224-bc7feb93edbd admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:52.775668 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c4a55c-3750-4ce9-a224-bc7feb93edbd admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:52.829522 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:52.851454 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba569499-37cf-4112-b043-267aca988ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:52.852197 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba569499-37cf-4112-b043-267aca988ffd admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:52.852485 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba569499-37cf-4112-b043-267aca988ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:52.852760 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba569499-37cf-4112-b043-267aca988ffd admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:52.853400 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba569499-37cf-4112-b043-267aca988ffd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:52.855941 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba569499-37cf-4112-b043-267aca988ffd admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:52.869967 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 871/1726] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:45:52 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 08:45:52.871010 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d61555-8303-4f85-be4d-eed51f3a695f admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:52.887297 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d61555-8303-4f85-be4d-eed51f3a695f admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:52.892190 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d4f371-0622-425e-8f9c-5972fd5582cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:52.892851 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d4f371-0622-425e-8f9c-5972fd5582cd admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:52.893153 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d4f371-0622-425e-8f9c-5972fd5582cd admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:52.893428 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d4f371-0622-425e-8f9c-5972fd5582cd admin admin] PATH_INFO: `/v3/projects/68cdf36f2d8e4f0c8f8d5a5d6c81e538/users/ac93b6c6e7be4cd185dfd824d864a1af/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:52.895374 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d4f371-0622-425e-8f9c-5972fd5582cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=68cdf36f2d8e4f0c8f8d5a5d6c81e538, user_id=ac93b6c6e7be4cd185dfd824d864a1af, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:52.898225 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d4f371-0622-425e-8f9c-5972fd5582cd admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:52.909739 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 856/1727] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:45:52 2022] PUT /identity/v3/projects/68cdf36f2d8e4f0c8f8d5a5d6c81e538/users/ac93b6c6e7be4cd185dfd824d864a1af/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:52.920223 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e382197d-858a-4bdf-8ee4-83c8c89bde11 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:52.920515 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e382197d-858a-4bdf-8ee4-83c8c89bde11 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:52.920845 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e382197d-858a-4bdf-8ee4-83c8c89bde11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:52.955426 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-e382197d-858a-4bdf-8ee4-83c8c89bde11 None None] MFA Rules not processed for user `ac93b6c6e7be4cd185dfd824d864a1af`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:52.956749 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-e382197d-858a-4bdf-8ee4-83c8c89bde11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:53.054087 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 872/1728] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:52 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:45:53.102388 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d4f371-0622-425e-8f9c-5972fd5582cd admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:53.104223 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-20d4f371-0622-425e-8f9c-5972fd5582cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:53.184272 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d4f371-0622-425e-8f9c-5972fd5582cd admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:53.187892 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdd929e-404d-45ac-b1e9-71310a9f3b70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:53.188501 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd929e-404d-45ac-b1e9-71310a9f3b70 service cinder] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:53.188782 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd929e-404d-45ac-b1e9-71310a9f3b70 service cinder] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:53.189059 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd929e-404d-45ac-b1e9-71310a9f3b70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:53.218424 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:53.258576 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdd929e-404d-45ac-b1e9-71310a9f3b70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:53.260016 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdd929e-404d-45ac-b1e9-71310a9f3b70 service cinder] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:53.269786 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 857/1729] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:45:53 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:45:54.851789 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e382197d-858a-4bdf-8ee4-83c8c89bde11 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:54.853328 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-e382197d-858a-4bdf-8ee4-83c8c89bde11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:54.866548 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e382197d-858a-4bdf-8ee4-83c8c89bde11 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:54.872174 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a76f8f8-c91d-476c-a939-f50ca388c6cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:54.872886 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a76f8f8-c91d-476c-a939-f50ca388c6cf service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:54.873237 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a76f8f8-c91d-476c-a939-f50ca388c6cf service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:54.873586 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a76f8f8-c91d-476c-a939-f50ca388c6cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:54.949695 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a76f8f8-c91d-476c-a939-f50ca388c6cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:54.951269 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a76f8f8-c91d-476c-a939-f50ca388c6cf service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:54.966930 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 873/1730] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:45:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:45:55.546614 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdd929e-404d-45ac-b1e9-71310a9f3b70 service cinder] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:55.555818 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdd929e-404d-45ac-b1e9-71310a9f3b70 service cinder] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:55.559518 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b258027b-88fd-45bb-9610-8fa4b681d2a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:55.560130 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b258027b-88fd-45bb-9610-8fa4b681d2a6 service cinder] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:55.560416 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b258027b-88fd-45bb-9610-8fa4b681d2a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:55.560694 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b258027b-88fd-45bb-9610-8fa4b681d2a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:55.570488 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b258027b-88fd-45bb-9610-8fa4b681d2a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:55.572102 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b258027b-88fd-45bb-9610-8fa4b681d2a6 service cinder] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:55.586531 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 858/1731] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:45:55 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:45:58.364769 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a76f8f8-c91d-476c-a939-f50ca388c6cf service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:58.366894 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-3a76f8f8-c91d-476c-a939-f50ca388c6cf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:58.490005 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a76f8f8-c91d-476c-a939-f50ca388c6cf service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:58.498436 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f6ae17-02bb-4330-a5be-aae250f8e000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:58.498436 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f6ae17-02bb-4330-a5be-aae250f8e000 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:58.498852 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f6ae17-02bb-4330-a5be-aae250f8e000 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:58.499851 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f6ae17-02bb-4330-a5be-aae250f8e000 admin admin] PATH_INFO: `/v3/users/bf66bc4c78b94c51b21097e460fa45d8` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:58.500859 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f6ae17-02bb-4330-a5be-aae250f8e000 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf66bc4c78b94c51b21097e460fa45d8)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:58.506100 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f6ae17-02bb-4330-a5be-aae250f8e000 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:58.514759 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:58.537459 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a4cc23-2987-419b-afd0-9ff492363208 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:58.538978 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a4cc23-2987-419b-afd0-9ff492363208 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:58.539731 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a4cc23-2987-419b-afd0-9ff492363208 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:58.541450 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 859/1732] 10.222.0.30 () {54 vars in 921 bytes} [Sat Oct 15 08:45:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:45:58.567361 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a8dd94-81a5-467d-b0bc-c0984b94c240 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:58.567642 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a8dd94-81a5-467d-b0bc-c0984b94c240 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:58.567921 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a8dd94-81a5-467d-b0bc-c0984b94c240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:58.571478 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-68f6ae17-02bb-4330-a5be-aae250f8e000 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf66bc4c78b94c51b21097e460fa45d8'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:58.572207 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-68f6ae17-02bb-4330-a5be-aae250f8e000 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf66bc4c78b94c51b21097e460fa45d8'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:58.581594 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-68f6ae17-02bb-4330-a5be-aae250f8e000 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf66bc4c78b94c51b21097e460fa45d8'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:58.591341 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-06a8dd94-81a5-467d-b0bc-c0984b94c240 None None] MFA Rules not processed for user `38986ca2612c4fc59d960cd693834f9b`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:58.594704 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-06a8dd94-81a5-467d-b0bc-c0984b94c240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:58.595490 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-68f6ae17-02bb-4330-a5be-aae250f8e000 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf66bc4c78b94c51b21097e460fa45d8'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:58.606481 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-68f6ae17-02bb-4330-a5be-aae250f8e000 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf66bc4c78b94c51b21097e460fa45d8'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:58.621928 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 874/1733] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 08:45:58 2022] DELETE /identity/v3/users/bf66bc4c78b94c51b21097e460fa45d8 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:58.624786 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f6ae17-02bb-4330-a5be-aae250f8e000 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:58.624786 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-68f6ae17-02bb-4330-a5be-aae250f8e000 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:58.685214 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 860/1734] 10.222.0.30 () {58 vars in 1018 bytes} [Sat Oct 15 08:45:58 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 08:45:58.691220 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b87ae2-431e-4ba1-aedf-a619b9ae4b40 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:58.691509 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b87ae2-431e-4ba1-aedf-a619b9ae4b40 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:58.691717 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b87ae2-431e-4ba1-aedf-a619b9ae4b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:58.698771 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f6ae17-02bb-4330-a5be-aae250f8e000 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:58.703609 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cf52c6-9a97-44ed-999b-e8864916ba22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:58.704246 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf52c6-9a97-44ed-999b-e8864916ba22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:58.704651 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf52c6-9a97-44ed-999b-e8864916ba22 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:58.704932 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf52c6-9a97-44ed-999b-e8864916ba22 admin admin] PATH_INFO: `/v3/users/2b1999640b3c41beafefe80d40bf965c` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:58.706101 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cf52c6-9a97-44ed-999b-e8864916ba22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b1999640b3c41beafefe80d40bf965c)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:58.708464 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cf52c6-9a97-44ed-999b-e8864916ba22 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:58.711660 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-91b87ae2-431e-4ba1-aedf-a619b9ae4b40 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:58.712856 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-91b87ae2-431e-4ba1-aedf-a619b9ae4b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:58.720523 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 861/1735] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:45:58.721340 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b87ae2-431e-4ba1-aedf-a619b9ae4b40 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:58.722622 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-91b87ae2-431e-4ba1-aedf-a619b9ae4b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:58.769290 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-04cf52c6-9a97-44ed-999b-e8864916ba22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b1999640b3c41beafefe80d40bf965c'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:58.769993 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-04cf52c6-9a97-44ed-999b-e8864916ba22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b1999640b3c41beafefe80d40bf965c'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:58.777934 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-04cf52c6-9a97-44ed-999b-e8864916ba22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b1999640b3c41beafefe80d40bf965c'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:58.791393 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-04cf52c6-9a97-44ed-999b-e8864916ba22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b1999640b3c41beafefe80d40bf965c'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:58.800841 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-04cf52c6-9a97-44ed-999b-e8864916ba22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b1999640b3c41beafefe80d40bf965c'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:58.803219 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:58.809227 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 875/1736] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 08:45:58 2022] DELETE /identity/v3/users/2b1999640b3c41beafefe80d40bf965c => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:58.810264 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cf52c6-9a97-44ed-999b-e8864916ba22 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:58.811648 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-04cf52c6-9a97-44ed-999b-e8864916ba22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:58.817292 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b87ae2-431e-4ba1-aedf-a619b9ae4b40 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:58.880299 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:58.886593 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cf52c6-9a97-44ed-999b-e8864916ba22 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:58.890879 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0328fd0b-a26f-4099-8eef-d3257eec38b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:58.891571 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0328fd0b-a26f-4099-8eef-d3257eec38b9 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:58.892338 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0328fd0b-a26f-4099-8eef-d3257eec38b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:58.892338 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0328fd0b-a26f-4099-8eef-d3257eec38b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:58.894278 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-0328fd0b-a26f-4099-8eef-d3257eec38b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:58.901161 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb28f70a-fa4f-4407-b949-60df5b7b1d41 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38986ca2612c4fc59d960cd693834f9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:58.902050 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb28f70a-fa4f-4407-b949-60df5b7b1d41 service placement] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:58.902337 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb28f70a-fa4f-4407-b949-60df5b7b1d41 service placement] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:58.902615 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb28f70a-fa4f-4407-b949-60df5b7b1d41 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:58.904413 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-fb28f70a-fa4f-4407-b949-60df5b7b1d41 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:58.917440 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb28f70a-fa4f-4407-b949-60df5b7b1d41 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:58.919606 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb28f70a-fa4f-4407-b949-60df5b7b1d41 service placement] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:58.934382 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 862/1737] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:45:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:45:58.964922 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0328fd0b-a26f-4099-8eef-d3257eec38b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:58.966603 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0328fd0b-a26f-4099-8eef-d3257eec38b9 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:58.976990 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 876/1738] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:45:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:45:59.048326 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb28f70a-fa4f-4407-b949-60df5b7b1d41 service placement] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:59.049928 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-fb28f70a-fa4f-4407-b949-60df5b7b1d41 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:59.064093 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb28f70a-fa4f-4407-b949-60df5b7b1d41 service placement] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:59.068936 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c18fc98-d97d-4843-9672-0c2b4f13607b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:59.069667 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18fc98-d97d-4843-9672-0c2b4f13607b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:59.069876 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18fc98-d97d-4843-9672-0c2b4f13607b admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:59.070157 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18fc98-d97d-4843-9672-0c2b4f13607b admin admin] PATH_INFO: `/v3/projects/bccc9c2f8b8f4bc3a4dbc121a32d82ad` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:59.071726 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c18fc98-d97d-4843-9672-0c2b4f13607b admin admin] RBAC: Authorizing `identity:delete_project(project_id=bccc9c2f8b8f4bc3a4dbc121a32d82ad)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:59.073837 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c18fc98-d97d-4843-9672-0c2b4f13607b admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:59.135345 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-8c18fc98-d97d-4843-9672-0c2b4f13607b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bccc9c2f8b8f4bc3a4dbc121a32d82ad', 'request_id': 'req-8c18fc98-d97d-4843-9672-0c2b4f13607b'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:59.142150 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-8c18fc98-d97d-4843-9672-0c2b4f13607b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bccc9c2f8b8f4bc3a4dbc121a32d82ad', 'request_id': 'req-8c18fc98-d97d-4843-9672-0c2b4f13607b'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:59.158378 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-8c18fc98-d97d-4843-9672-0c2b4f13607b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bccc9c2f8b8f4bc3a4dbc121a32d82ad', 'request_id': 'req-8c18fc98-d97d-4843-9672-0c2b4f13607b'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:59.171913 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-8c18fc98-d97d-4843-9672-0c2b4f13607b admin admin] The token cache is being invalidate because project bccc9c2f8b8f4bc3a4dbc121a32d82ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 08:45:59.172234 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-8c18fc98-d97d-4843-9672-0c2b4f13607b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:59.172903 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-8c18fc98-d97d-4843-9672-0c2b4f13607b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:59.174476 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 863/1739] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 08:45:59 2022] DELETE /identity/v3/projects/bccc9c2f8b8f4bc3a4dbc121a32d82ad => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:59.179588 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894194a7-b859-4e28-9ee0-12706abd3364 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:59.179884 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894194a7-b859-4e28-9ee0-12706abd3364 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:59.180169 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894194a7-b859-4e28-9ee0-12706abd3364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:59.200484 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-894194a7-b859-4e28-9ee0-12706abd3364 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:59.201902 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-894194a7-b859-4e28-9ee0-12706abd3364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:59.264167 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 877/1740] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 08:45:59.272475 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77540f3b-f2af-4905-87fb-26bba1b07005 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:59.272947 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77540f3b-f2af-4905-87fb-26bba1b07005 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:59.273232 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77540f3b-f2af-4905-87fb-26bba1b07005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:59.287413 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894194a7-b859-4e28-9ee0-12706abd3364 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:59.289406 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-894194a7-b859-4e28-9ee0-12706abd3364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:59.297144 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-77540f3b-f2af-4905-87fb-26bba1b07005 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:59.298387 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-77540f3b-f2af-4905-87fb-26bba1b07005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:59.302232 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894194a7-b859-4e28-9ee0-12706abd3364 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:59.307538 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554c26c3-da8c-45c5-8f7f-55f2d9640946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:59.308636 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 864/1741] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 08:45:59.309924 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c26c3-da8c-45c5-8f7f-55f2d9640946 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:59.310466 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c26c3-da8c-45c5-8f7f-55f2d9640946 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:59.310808 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c26c3-da8c-45c5-8f7f-55f2d9640946 admin admin] PATH_INFO: `/v3/projects/80799151ef6c4be8b1b436daebe8a98b` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:59.312631 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554c26c3-da8c-45c5-8f7f-55f2d9640946 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80799151ef6c4be8b1b436daebe8a98b)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:59.312756 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77540f3b-f2af-4905-87fb-26bba1b07005 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:59.314627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554c26c3-da8c-45c5-8f7f-55f2d9640946 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:59.322706 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77540f3b-f2af-4905-87fb-26bba1b07005 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:59.327430 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0030a51d-0579-40fe-9f0b-1977e360914c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:59.328064 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0030a51d-0579-40fe-9f0b-1977e360914c admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:59.328341 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0030a51d-0579-40fe-9f0b-1977e360914c admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:59.328615 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0030a51d-0579-40fe-9f0b-1977e360914c admin admin] PATH_INFO: `/v3/domains` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:59.329260 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0030a51d-0579-40fe-9f0b-1977e360914c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:45:59.329710 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0030a51d-0579-40fe-9f0b-1977e360914c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:59.331230 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0030a51d-0579-40fe-9f0b-1977e360914c admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:59.343904 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 865/1742] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 08:45:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 08:45:59.348486 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0030a51d-0579-40fe-9f0b-1977e360914c admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:59.359748 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0030a51d-0579-40fe-9f0b-1977e360914c admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:59.361982 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:59.365328 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-554c26c3-da8c-45c5-8f7f-55f2d9640946 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80799151ef6c4be8b1b436daebe8a98b', 'request_id': 'req-554c26c3-da8c-45c5-8f7f-55f2d9640946'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:59.372890 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-554c26c3-da8c-45c5-8f7f-55f2d9640946 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80799151ef6c4be8b1b436daebe8a98b', 'request_id': 'req-554c26c3-da8c-45c5-8f7f-55f2d9640946'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:59.380401 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-554c26c3-da8c-45c5-8f7f-55f2d9640946 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80799151ef6c4be8b1b436daebe8a98b', 'request_id': 'req-554c26c3-da8c-45c5-8f7f-55f2d9640946'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:59.393983 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-554c26c3-da8c-45c5-8f7f-55f2d9640946 admin admin] The token cache is being invalidate because project 80799151ef6c4be8b1b436daebe8a98b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 08:45:59.399920 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-554c26c3-da8c-45c5-8f7f-55f2d9640946 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:59.399920 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-554c26c3-da8c-45c5-8f7f-55f2d9640946 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:59.399920 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 878/1743] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 08:45:59 2022] DELETE /identity/v3/projects/80799151ef6c4be8b1b436daebe8a98b => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 08:45:59.403010 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4b812f-39a8-4e79-88a1-2537495e2d54 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:59.403010 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4b812f-39a8-4e79-88a1-2537495e2d54 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:59.403145 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4b812f-39a8-4e79-88a1-2537495e2d54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:59.419577 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b294db69-b720-4bb5-b5f6-47644f5029e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:59.420192 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b294db69-b720-4bb5-b5f6-47644f5029e2 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:59.420478 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b294db69-b720-4bb5-b5f6-47644f5029e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:59.420758 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b294db69-b720-4bb5-b5f6-47644f5029e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:59.421401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b294db69-b720-4bb5-b5f6-47644f5029e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:59.423030 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-ee4b812f-39a8-4e79-88a1-2537495e2d54 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:59.423349 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b294db69-b720-4bb5-b5f6-47644f5029e2 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:59.424499 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-ee4b812f-39a8-4e79-88a1-2537495e2d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:59.432922 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 879/1744] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:45:59.441870 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae1e77-5f14-4a99-a199-5d09f5135d39 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:59.442284 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae1e77-5f14-4a99-a199-5d09f5135d39 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:59.442607 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae1e77-5f14-4a99-a199-5d09f5135d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:59.452933 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 866/1745] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 08:45:59 2022] POST /identity/v3/projects => generated 363 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 08:45:59.457031 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b294db69-b720-4bb5-b5f6-47644f5029e2 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:59.458768 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-b294db69-b720-4bb5-b5f6-47644f5029e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:59.463376 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-f6ae1e77-5f14-4a99-a199-5d09f5135d39 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:59.464551 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-f6ae1e77-5f14-4a99-a199-5d09f5135d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:59.485243 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:59.527159 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b294db69-b720-4bb5-b5f6-47644f5029e2 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:59.527471 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 880/1746] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 08:45:59.531783 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae1e77-5f14-4a99-a199-5d09f5135d39 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:59.531924 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c688f6b-299d-43cf-99df-9021b16eda7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:59.532459 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c688f6b-299d-43cf-99df-9021b16eda7c admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:59.532737 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c688f6b-299d-43cf-99df-9021b16eda7c admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:59.533009 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c688f6b-299d-43cf-99df-9021b16eda7c admin admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:59.533632 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c688f6b-299d-43cf-99df-9021b16eda7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:59.535578 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c688f6b-299d-43cf-99df-9021b16eda7c admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:59.547777 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae1e77-5f14-4a99-a199-5d09f5135d39 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:59.547777 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fa2343-dabf-47a0-a05e-2a88e1680b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:59.548281 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa2343-dabf-47a0-a05e-2a88e1680b48 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:59.548625 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa2343-dabf-47a0-a05e-2a88e1680b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:59.549283 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa2343-dabf-47a0-a05e-2a88e1680b48 admin admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:59.549660 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fa2343-dabf-47a0-a05e-2a88e1680b48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:45:59.550019 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fa2343-dabf-47a0-a05e-2a88e1680b48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:59.551827 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fa2343-dabf-47a0-a05e-2a88e1680b48 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:59.559278 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 867/1747] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:45:59 2022] POST /identity/v3/users => generated 282 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 08:45:59.563116 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c688f6b-299d-43cf-99df-9021b16eda7c admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:59.565040 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 881/1748] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 08:45:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 08:45:59.569093 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fa2343-dabf-47a0-a05e-2a88e1680b48 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:59.577908 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c688f6b-299d-43cf-99df-9021b16eda7c admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:59.579271 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fa2343-dabf-47a0-a05e-2a88e1680b48 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:59.582980 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f640e5b0-16b9-410d-8422-08650fb9ee9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:59.583775 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f640e5b0-16b9-410d-8422-08650fb9ee9c admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:59.584102 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f640e5b0-16b9-410d-8422-08650fb9ee9c admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:59.584382 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f640e5b0-16b9-410d-8422-08650fb9ee9c admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:59.584661 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008fce44-cc82-4802-8dde-1aec8c7506eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:59.585262 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f640e5b0-16b9-410d-8422-08650fb9ee9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:59.585386 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008fce44-cc82-4802-8dde-1aec8c7506eb admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:59.585694 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008fce44-cc82-4802-8dde-1aec8c7506eb admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:59.586085 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008fce44-cc82-4802-8dde-1aec8c7506eb admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:59.586813 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008fce44-cc82-4802-8dde-1aec8c7506eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:59.586935 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f640e5b0-16b9-410d-8422-08650fb9ee9c admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:59.588836 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008fce44-cc82-4802-8dde-1aec8c7506eb admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:59.601578 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 868/1749] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:45:59 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 08:45:59.601578 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f640e5b0-16b9-410d-8422-08650fb9ee9c admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:59.601578 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-f640e5b0-16b9-410d-8422-08650fb9ee9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:59.620865 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f640e5b0-16b9-410d-8422-08650fb9ee9c admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:59.626039 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e14aa72-994d-4581-80c6-6556dd267c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:59.626682 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e14aa72-994d-4581-80c6-6556dd267c12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:59.626961 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e14aa72-994d-4581-80c6-6556dd267c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:59.627233 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e14aa72-994d-4581-80c6-6556dd267c12 admin admin] PATH_INFO: `/v3/users/8caf111ba96d4b67bcd466eaa03c52e0` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:59.628405 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e14aa72-994d-4581-80c6-6556dd267c12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8caf111ba96d4b67bcd466eaa03c52e0)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:59.632281 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 882/1750] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 08:45:59 2022] POST /identity/v3/projects => generated 383 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:45:59.632281 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008fce44-cc82-4802-8dde-1aec8c7506eb admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:59.632492 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e14aa72-994d-4581-80c6-6556dd267c12 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:59.679261 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-7e14aa72-994d-4581-80c6-6556dd267c12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8caf111ba96d4b67bcd466eaa03c52e0'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:59.679930 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-7e14aa72-994d-4581-80c6-6556dd267c12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8caf111ba96d4b67bcd466eaa03c52e0'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:59.692165 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-7e14aa72-994d-4581-80c6-6556dd267c12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8caf111ba96d4b67bcd466eaa03c52e0'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:59.697271 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008fce44-cc82-4802-8dde-1aec8c7506eb admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:59.699520 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-7e14aa72-994d-4581-80c6-6556dd267c12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8caf111ba96d4b67bcd466eaa03c52e0'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:59.700174 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-7e14aa72-994d-4581-80c6-6556dd267c12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8caf111ba96d4b67bcd466eaa03c52e0'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:45:59.702125 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2029906-cd9d-4090-8dac-b1cf650da291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:59.702732 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2029906-cd9d-4090-8dac-b1cf650da291 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:59.703008 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2029906-cd9d-4090-8dac-b1cf650da291 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:59.703279 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2029906-cd9d-4090-8dac-b1cf650da291 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:59.703919 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2029906-cd9d-4090-8dac-b1cf650da291 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:59.705500 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2029906-cd9d-4090-8dac-b1cf650da291 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:59.713192 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 869/1751] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 08:45:59 2022] DELETE /identity/v3/users/8caf111ba96d4b67bcd466eaa03c52e0 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:59.714019 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e14aa72-994d-4581-80c6-6556dd267c12 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:59.715774 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-7e14aa72-994d-4581-80c6-6556dd267c12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:59.715893 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 883/1752] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:45:59 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:45:59.720500 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b0dfce-6cfa-4d37-8085-1eaafd6dc114 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:59.720789 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b0dfce-6cfa-4d37-8085-1eaafd6dc114 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:59.721127 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b0dfce-6cfa-4d37-8085-1eaafd6dc114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:59.741388 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-a2b0dfce-6cfa-4d37-8085-1eaafd6dc114 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:59.742637 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-a2b0dfce-6cfa-4d37-8085-1eaafd6dc114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:59.766647 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:59.779448 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e14aa72-994d-4581-80c6-6556dd267c12 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:59.784041 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4223f2a-c15a-4a5a-b0c3-fab3e1bc5feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:59.784653 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4223f2a-c15a-4a5a-b0c3-fab3e1bc5feb admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:59.784931 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4223f2a-c15a-4a5a-b0c3-fab3e1bc5feb admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:59.785208 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4223f2a-c15a-4a5a-b0c3-fab3e1bc5feb admin admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:59.785867 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4223f2a-c15a-4a5a-b0c3-fab3e1bc5feb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:59.788322 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4223f2a-c15a-4a5a-b0c3-fab3e1bc5feb admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:59.804064 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 884/1753] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 08:45:59.804891 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b0dfce-6cfa-4d37-8085-1eaafd6dc114 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:59.806201 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-a2b0dfce-6cfa-4d37-8085-1eaafd6dc114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:59.815084 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 870/1754] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:45:59 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:45:59.815084 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4223f2a-c15a-4a5a-b0c3-fab3e1bc5feb admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:59.817253 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-b4223f2a-c15a-4a5a-b0c3-fab3e1bc5feb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:45:59.820052 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b0dfce-6cfa-4d37-8085-1eaafd6dc114 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:59.823337 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=83,keystone:INSERT=7,keystone:DELETE=10 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:45:59.825516 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f8da4b-3946-4ee8-8e7e-358a64f8ea2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:59.826296 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f8da4b-3946-4ee8-8e7e-358a64f8ea2a admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:59.826581 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f8da4b-3946-4ee8-8e7e-358a64f8ea2a admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:59.826875 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f8da4b-3946-4ee8-8e7e-358a64f8ea2a admin admin] PATH_INFO: `/v3/projects/36ed3386378e4de49a4ede7466d1decd/users/e3713cd231374c4d93689823d3f1562b/roles/3248f0268738489b883041851d681bb7` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:59.842886 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f8da4b-3946-4ee8-8e7e-358a64f8ea2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=36ed3386378e4de49a4ede7466d1decd, user_id=e3713cd231374c4d93689823d3f1562b, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:59.845561 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f8da4b-3946-4ee8-8e7e-358a64f8ea2a admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:59.859578 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 885/1755] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:45:59 2022] PUT /identity/v3/projects/36ed3386378e4de49a4ede7466d1decd/users/e3713cd231374c4d93689823d3f1562b/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:45:59.869776 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f8da4b-3946-4ee8-8e7e-358a64f8ea2a admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:59.893074 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4223f2a-c15a-4a5a-b0c3-fab3e1bc5feb admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:59.941383 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f8da4b-3946-4ee8-8e7e-358a64f8ea2a admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:45:59.947981 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4776f3b1-de6f-44fc-b303-a249b910835d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:59.949070 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4776f3b1-de6f-44fc-b303-a249b910835d admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:59.949363 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4776f3b1-de6f-44fc-b303-a249b910835d admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:59.949733 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4776f3b1-de6f-44fc-b303-a249b910835d admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:59.950401 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4776f3b1-de6f-44fc-b303-a249b910835d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:59.952057 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4776f3b1-de6f-44fc-b303-a249b910835d admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:59.956921 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0a42cc-6497-47f5-aedb-f2c32147419d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:45:59.957540 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a42cc-6497-47f5-aedb-f2c32147419d service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:59.957848 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a42cc-6497-47f5-aedb-f2c32147419d service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:59.958678 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a42cc-6497-47f5-aedb-f2c32147419d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:59.964377 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 886/1756] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:45:59 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:45:59.965644 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3380beeb-3388-4c1e-b180-88baac07ba03 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:45:59.966780 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3380beeb-3388-4c1e-b180-88baac07ba03 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:45:59.967286 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3380beeb-3388-4c1e-b180-88baac07ba03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:45:59.973183 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0a42cc-6497-47f5-aedb-f2c32147419d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:45:59.975002 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0a42cc-6497-47f5-aedb-f2c32147419d service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:45:59.987260 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 871/1757] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:45:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:45:59.988596 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0a42cc-6497-47f5-aedb-f2c32147419d service neutron] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:45:59.997618 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-3380beeb-3388-4c1e-b180-88baac07ba03 None None] MFA Rules not processed for user `e3713cd231374c4d93689823d3f1562b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:45:59.998974 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-3380beeb-3388-4c1e-b180-88baac07ba03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:46:00.000674 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0a42cc-6497-47f5-aedb-f2c32147419d service neutron] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:00.005435 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afb8cf2-9c08-42c1-881e-e5643db51986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:00.006113 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb8cf2-9c08-42c1-881e-e5643db51986 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:00.006664 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb8cf2-9c08-42c1-881e-e5643db51986 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:00.007033 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb8cf2-9c08-42c1-881e-e5643db51986 admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:00.007585 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afb8cf2-9c08-42c1-881e-e5643db51986 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:00.009394 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afb8cf2-9c08-42c1-881e-e5643db51986 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:00.023288 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 872/1758] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:45:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:46:00.027203 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afb8cf2-9c08-42c1-881e-e5643db51986 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:00.045917 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afb8cf2-9c08-42c1-881e-e5643db51986 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:00.052836 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26da1f2-beaa-425e-a97a-67e20a19b577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:00.053996 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26da1f2-beaa-425e-a97a-67e20a19b577 admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:00.053996 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26da1f2-beaa-425e-a97a-67e20a19b577 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:00.054262 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26da1f2-beaa-425e-a97a-67e20a19b577 admin admin] PATH_INFO: `/v3/projects/6c350a7b4ae944c188fc8af5a91438f3/users/d716aed1f7684cda9b82ba0d5c3d24d4/roles/3248f0268738489b883041851d681bb7` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:00.075778 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26da1f2-beaa-425e-a97a-67e20a19b577 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c350a7b4ae944c188fc8af5a91438f3, user_id=d716aed1f7684cda9b82ba0d5c3d24d4, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:00.078828 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26da1f2-beaa-425e-a97a-67e20a19b577 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:00.089496 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 887/1759] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:45:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:46:00.094039 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3380beeb-3388-4c1e-b180-88baac07ba03 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:00.097142 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 873/1760] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:46:00 2022] PUT /identity/v3/projects/6c350a7b4ae944c188fc8af5a91438f3/users/d716aed1f7684cda9b82ba0d5c3d24d4/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:46:00.098561 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a564f-f21e-417f-b806-ab54c16b3057 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:00.098849 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a564f-f21e-417f-b806-ab54c16b3057 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:00.099124 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a564f-f21e-417f-b806-ab54c16b3057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:00.105992 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3380beeb-3388-4c1e-b180-88baac07ba03 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:00.120489 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-551a564f-f21e-417f-b806-ab54c16b3057 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:46:00.121826 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-551a564f-f21e-417f-b806-ab54c16b3057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:46:00.146139 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:46:00.187563 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 874/1761] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:46:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:46:00.187866 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a6c4ff-524f-481b-905f-a73ffcc446d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:00.188484 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a6c4ff-524f-481b-905f-a73ffcc446d9 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:00.188768 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a6c4ff-524f-481b-905f-a73ffcc446d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:00.189045 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a6c4ff-524f-481b-905f-a73ffcc446d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:00.189803 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a6c4ff-524f-481b-905f-a73ffcc446d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:00.191948 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a6c4ff-524f-481b-905f-a73ffcc446d9 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:00.197536 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551a564f-f21e-417f-b806-ab54c16b3057 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:00.227663 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 888/1762] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 08:46:00 2022] POST /identity/v3/projects => generated 363 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 08:46:00.229389 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a6c4ff-524f-481b-905f-a73ffcc446d9 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:00.234315 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:46:00.260705 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551a564f-f21e-417f-b806-ab54c16b3057 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:00.303618 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a6c4ff-524f-481b-905f-a73ffcc446d9 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:00.312320 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aedc76-0f18-41fb-b843-28770c8d622d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:00.313209 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aedc76-0f18-41fb-b843-28770c8d622d admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:00.313732 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aedc76-0f18-41fb-b843-28770c8d622d admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:00.314201 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aedc76-0f18-41fb-b843-28770c8d622d admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:00.315033 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aedc76-0f18-41fb-b843-28770c8d622d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:00.316730 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a86ea67-c1e8-442f-a4c8-50b9baac30cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:00.317380 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a86ea67-c1e8-442f-a4c8-50b9baac30cf service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:00.317667 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a86ea67-c1e8-442f-a4c8-50b9baac30cf service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:00.317854 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aedc76-0f18-41fb-b843-28770c8d622d admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:00.317965 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a86ea67-c1e8-442f-a4c8-50b9baac30cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:00.330433 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a86ea67-c1e8-442f-a4c8-50b9baac30cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:00.332532 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a86ea67-c1e8-442f-a4c8-50b9baac30cf service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:00.343101 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 889/1763] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:46:00 2022] POST /identity/v3/users => generated 288 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:46:00.343908 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aedc76-0f18-41fb-b843-28770c8d622d admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:00.344671 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 875/1764] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:46:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 08:46:00.347116 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a86ea67-c1e8-442f-a4c8-50b9baac30cf service neutron] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:00.348424 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-f7aedc76-0f18-41fb-b843-28770c8d622d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:46:00.356682 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a86ea67-c1e8-442f-a4c8-50b9baac30cf service neutron] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:00.360693 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aedc76-0f18-41fb-b843-28770c8d622d admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:00.365605 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af66e35-b371-4d9a-bef1-69104296edaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:00.366396 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af66e35-b371-4d9a-bef1-69104296edaa admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:00.366682 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af66e35-b371-4d9a-bef1-69104296edaa admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:00.366983 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af66e35-b371-4d9a-bef1-69104296edaa admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:00.367630 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af66e35-b371-4d9a-bef1-69104296edaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:00.368063 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be45c5e-03fb-4a79-ba4b-06c234b19486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:00.368859 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be45c5e-03fb-4a79-ba4b-06c234b19486 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:00.369167 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be45c5e-03fb-4a79-ba4b-06c234b19486 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:00.369269 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af66e35-b371-4d9a-bef1-69104296edaa admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:00.369483 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be45c5e-03fb-4a79-ba4b-06c234b19486 admin admin] PATH_INFO: `/v3/projects/4377987106334f2f8dcc53977ba35fd7` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:00.370871 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be45c5e-03fb-4a79-ba4b-06c234b19486 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4377987106334f2f8dcc53977ba35fd7)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:00.373330 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be45c5e-03fb-4a79-ba4b-06c234b19486 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:00.380087 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 876/1765] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:46:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:46:00.384675 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af66e35-b371-4d9a-bef1-69104296edaa admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:00.395927 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af66e35-b371-4d9a-bef1-69104296edaa admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:00.402079 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a193666-9833-49cd-96c1-4fec25d003c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:00.402753 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a193666-9833-49cd-96c1-4fec25d003c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:00.403470 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a193666-9833-49cd-96c1-4fec25d003c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:00.403470 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a193666-9833-49cd-96c1-4fec25d003c4 admin admin] PATH_INFO: `/v3/projects/5450acaf9fc94ca0baad9a74c42b3652/users/8d0f3b24ec7f4525a2a086d95dbd8395/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:00.420594 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a193666-9833-49cd-96c1-4fec25d003c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5450acaf9fc94ca0baad9a74c42b3652, user_id=8d0f3b24ec7f4525a2a086d95dbd8395, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:00.424031 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a193666-9833-49cd-96c1-4fec25d003c4 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:00.434532 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-5be45c5e-03fb-4a79-ba4b-06c234b19486 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4377987106334f2f8dcc53977ba35fd7', 'request_id': 'req-5be45c5e-03fb-4a79-ba4b-06c234b19486'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:00.440004 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 877/1766] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:46:00 2022] PUT /identity/v3/projects/5450acaf9fc94ca0baad9a74c42b3652/users/8d0f3b24ec7f4525a2a086d95dbd8395/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 08:46:00.444575 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a193666-9833-49cd-96c1-4fec25d003c4 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:00.444709 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-5be45c5e-03fb-4a79-ba4b-06c234b19486 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4377987106334f2f8dcc53977ba35fd7', 'request_id': 'req-5be45c5e-03fb-4a79-ba4b-06c234b19486'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:00.453705 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-5be45c5e-03fb-4a79-ba4b-06c234b19486 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4377987106334f2f8dcc53977ba35fd7', 'request_id': 'req-5be45c5e-03fb-4a79-ba4b-06c234b19486'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:00.468399 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-5be45c5e-03fb-4a79-ba4b-06c234b19486 admin admin] The token cache is being invalidate because project 4377987106334f2f8dcc53977ba35fd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 08:46:00.468698 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-5be45c5e-03fb-4a79-ba4b-06c234b19486 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:00.469953 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-5be45c5e-03fb-4a79-ba4b-06c234b19486 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:00.474077 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 890/1767] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 08:46:00 2022] DELETE /identity/v3/projects/4377987106334f2f8dcc53977ba35fd7 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:46:00.475539 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be45c5e-03fb-4a79-ba4b-06c234b19486 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:00.477420 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-5be45c5e-03fb-4a79-ba4b-06c234b19486 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:46:00.515834 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:46:00.531551 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a193666-9833-49cd-96c1-4fec25d003c4 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:00.538058 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6671b6a7-afc6-45d9-868d-8e7072da95ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:00.538736 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6671b6a7-afc6-45d9-868d-8e7072da95ef admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:00.539027 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6671b6a7-afc6-45d9-868d-8e7072da95ef admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:00.539465 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6671b6a7-afc6-45d9-868d-8e7072da95ef admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:00.540015 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6671b6a7-afc6-45d9-868d-8e7072da95ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:00.542081 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6671b6a7-afc6-45d9-868d-8e7072da95ef admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:00.555097 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 878/1768] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:46:00 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:46:00.559873 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6671b6a7-afc6-45d9-868d-8e7072da95ef admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:00.562160 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-6671b6a7-afc6-45d9-868d-8e7072da95ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:46:00.564528 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be45c5e-03fb-4a79-ba4b-06c234b19486 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:00.571386 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594c44cf-f605-4db0-897a-a4bd3e516a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:00.571764 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c44cf-f605-4db0-897a-a4bd3e516a40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:00.572085 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c44cf-f605-4db0-897a-a4bd3e516a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:00.573276 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c44cf-f605-4db0-897a-a4bd3e516a40 admin admin] PATH_INFO: `/v3/users/15105cfd810945baadb52bd4cd1ec04b` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:00.573607 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594c44cf-f605-4db0-897a-a4bd3e516a40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15105cfd810945baadb52bd4cd1ec04b)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:00.575636 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594c44cf-f605-4db0-897a-a4bd3e516a40 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:00.576522 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6671b6a7-afc6-45d9-868d-8e7072da95ef admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:00.582012 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817e53a0-e53c-4ff3-83c6-393cce0b610f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:00.582648 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817e53a0-e53c-4ff3-83c6-393cce0b610f admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:00.583026 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817e53a0-e53c-4ff3-83c6-393cce0b610f admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:00.583361 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817e53a0-e53c-4ff3-83c6-393cce0b610f admin admin] PATH_INFO: `/v3/projects/5450acaf9fc94ca0baad9a74c42b3652/users/8d0f3b24ec7f4525a2a086d95dbd8395/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:00.585942 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817e53a0-e53c-4ff3-83c6-393cce0b610f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5450acaf9fc94ca0baad9a74c42b3652, user_id=8d0f3b24ec7f4525a2a086d95dbd8395, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:00.588638 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817e53a0-e53c-4ff3-83c6-393cce0b610f admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:00.599688 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 879/1769] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:46:00 2022] PUT /identity/v3/projects/5450acaf9fc94ca0baad9a74c42b3652/users/8d0f3b24ec7f4525a2a086d95dbd8395/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 08:46:00.614008 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6d166b-4b2b-474c-992e-2ce3ae377cf6 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:00.614477 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6d166b-4b2b-474c-992e-2ce3ae377cf6 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:00.614902 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6d166b-4b2b-474c-992e-2ce3ae377cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:00.636829 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-594c44cf-f605-4db0-897a-a4bd3e516a40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15105cfd810945baadb52bd4cd1ec04b'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:00.637783 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-594c44cf-f605-4db0-897a-a4bd3e516a40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15105cfd810945baadb52bd4cd1ec04b'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:00.646304 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-594c44cf-f605-4db0-897a-a4bd3e516a40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15105cfd810945baadb52bd4cd1ec04b'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:00.646461 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-cf6d166b-4b2b-474c-992e-2ce3ae377cf6 None None] MFA Rules not processed for user `8d0f3b24ec7f4525a2a086d95dbd8395`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:46:00.648254 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-cf6d166b-4b2b-474c-992e-2ce3ae377cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:46:00.667959 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-594c44cf-f605-4db0-897a-a4bd3e516a40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15105cfd810945baadb52bd4cd1ec04b'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:00.682014 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:46:00.689477 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-594c44cf-f605-4db0-897a-a4bd3e516a40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15105cfd810945baadb52bd4cd1ec04b'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:00.698665 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 891/1770] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 08:46:00 2022] DELETE /identity/v3/users/15105cfd810945baadb52bd4cd1ec04b => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 08:46:00.752751 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 880/1771] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:46:00 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:46:00.766107 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594c44cf-f605-4db0-897a-a4bd3e516a40 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:00.768027 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-594c44cf-f605-4db0-897a-a4bd3e516a40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:46:00.835132 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594c44cf-f605-4db0-897a-a4bd3e516a40 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:00.838876 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649fbaf9-3865-4904-9d0f-49731558e5e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:00.839571 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649fbaf9-3865-4904-9d0f-49731558e5e2 service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:00.839925 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649fbaf9-3865-4904-9d0f-49731558e5e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:00.840273 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649fbaf9-3865-4904-9d0f-49731558e5e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:00.914049 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649fbaf9-3865-4904-9d0f-49731558e5e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:00.916929 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649fbaf9-3865-4904-9d0f-49731558e5e2 service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:00.922560 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:46:00.935411 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 892/1772] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:46:00 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:46:01.004844 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6d166b-4b2b-474c-992e-2ce3ae377cf6 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:01.007783 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-cf6d166b-4b2b-474c-992e-2ce3ae377cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:46:01.051054 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649fbaf9-3865-4904-9d0f-49731558e5e2 service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:01.073440 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649fbaf9-3865-4904-9d0f-49731558e5e2 service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:01.080273 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ff739-f0b4-4113-820e-8e139bb0913f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:01.080911 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ff739-f0b4-4113-820e-8e139bb0913f service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:01.081193 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ff739-f0b4-4113-820e-8e139bb0913f service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:01.081558 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ff739-f0b4-4113-820e-8e139bb0913f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:01.086102 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-af7ff739-f0b4-4113-820e-8e139bb0913f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:46:01.115078 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6d166b-4b2b-474c-992e-2ce3ae377cf6 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:01.185099 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7ff739-f0b4-4113-820e-8e139bb0913f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:01.185775 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b533ba-abe1-4139-84a4-2d8f7a410988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:01.186951 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7ff739-f0b4-4113-820e-8e139bb0913f service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:01.187094 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b533ba-abe1-4139-84a4-2d8f7a410988 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:01.187440 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b533ba-abe1-4139-84a4-2d8f7a410988 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:01.187786 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b533ba-abe1-4139-84a4-2d8f7a410988 admin admin] PATH_INFO: `/v3/projects/4eece675165a483f95378350c3a5f759` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:01.189691 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b533ba-abe1-4139-84a4-2d8f7a410988 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4eece675165a483f95378350c3a5f759)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:01.194215 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b533ba-abe1-4139-84a4-2d8f7a410988 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:01.199350 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 893/1773] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:46:01 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:46:01.204920 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f8365-c17a-4f48-be31-52263fbaeab0 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:01.205510 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f8365-c17a-4f48-be31-52263fbaeab0 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:01.207183 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f8365-c17a-4f48-be31-52263fbaeab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:01.264845 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-49b533ba-abe1-4139-84a4-2d8f7a410988 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4eece675165a483f95378350c3a5f759', 'request_id': 'req-49b533ba-abe1-4139-84a4-2d8f7a410988'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:01.267467 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-d94f8365-c17a-4f48-be31-52263fbaeab0 None None] MFA Rules not processed for user `d716aed1f7684cda9b82ba0d5c3d24d4`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:46:01.269071 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-d94f8365-c17a-4f48-be31-52263fbaeab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:46:01.278476 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-49b533ba-abe1-4139-84a4-2d8f7a410988 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4eece675165a483f95378350c3a5f759', 'request_id': 'req-49b533ba-abe1-4139-84a4-2d8f7a410988'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:01.290676 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-49b533ba-abe1-4139-84a4-2d8f7a410988 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4eece675165a483f95378350c3a5f759', 'request_id': 'req-49b533ba-abe1-4139-84a4-2d8f7a410988'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:01.311475 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:46:01.316081 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-49b533ba-abe1-4139-84a4-2d8f7a410988 admin admin] The token cache is being invalidate because project 4eece675165a483f95378350c3a5f759 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 08:46:01.316081 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-49b533ba-abe1-4139-84a4-2d8f7a410988 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:01.316424 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-49b533ba-abe1-4139-84a4-2d8f7a410988 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:01.318449 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 881/1774] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 08:46:01 2022] DELETE /identity/v3/projects/4eece675165a483f95378350c3a5f759 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 08:46:01.322391 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b533ba-abe1-4139-84a4-2d8f7a410988 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:01.325878 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-49b533ba-abe1-4139-84a4-2d8f7a410988 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:46:01.384017 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 894/1775] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:46:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:46:01.396055 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94f8365-c17a-4f48-be31-52263fbaeab0 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:01.398494 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-d94f8365-c17a-4f48-be31-52263fbaeab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:46:01.433208 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b533ba-abe1-4139-84a4-2d8f7a410988 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:01.438349 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca1a6fc-fccb-4b66-9ea7-c9a78cc203f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:01.438994 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca1a6fc-fccb-4b66-9ea7-c9a78cc203f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:01.439882 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca1a6fc-fccb-4b66-9ea7-c9a78cc203f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:01.440172 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca1a6fc-fccb-4b66-9ea7-c9a78cc203f2 admin admin] PATH_INFO: `/v3/users/ac93b6c6e7be4cd185dfd824d864a1af` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:01.442045 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca1a6fc-fccb-4b66-9ea7-c9a78cc203f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac93b6c6e7be4cd185dfd824d864a1af)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:01.444060 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca1a6fc-fccb-4b66-9ea7-c9a78cc203f2 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:01.480568 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94f8365-c17a-4f48-be31-52263fbaeab0 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:01.484433 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abf0922-5103-4644-993d-5317a526abbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:01.485043 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf0922-5103-4644-993d-5317a526abbd service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:01.485329 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf0922-5103-4644-993d-5317a526abbd service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:01.485632 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf0922-5103-4644-993d-5317a526abbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:01.496065 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abf0922-5103-4644-993d-5317a526abbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:01.497549 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abf0922-5103-4644-993d-5317a526abbd service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:01.506687 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 895/1776] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:46:01 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 08:46:01.507903 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-9ca1a6fc-fccb-4b66-9ea7-c9a78cc203f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac93b6c6e7be4cd185dfd824d864a1af'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:01.508649 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-9ca1a6fc-fccb-4b66-9ea7-c9a78cc203f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac93b6c6e7be4cd185dfd824d864a1af'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:01.521465 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-9ca1a6fc-fccb-4b66-9ea7-c9a78cc203f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac93b6c6e7be4cd185dfd824d864a1af'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:01.530345 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-9ca1a6fc-fccb-4b66-9ea7-c9a78cc203f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac93b6c6e7be4cd185dfd824d864a1af'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:01.531073 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-9ca1a6fc-fccb-4b66-9ea7-c9a78cc203f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac93b6c6e7be4cd185dfd824d864a1af'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:01.544891 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 882/1777] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 08:46:01 2022] DELETE /identity/v3/users/ac93b6c6e7be4cd185dfd824d864a1af => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:46:01.841749 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abf0922-5103-4644-993d-5317a526abbd service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:01.862580 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abf0922-5103-4644-993d-5317a526abbd service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:01.891887 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:46:01.923441 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55d8a98-5e61-47ef-a49a-b8fff69e6a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:01.924072 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d8a98-5e61-47ef-a49a-b8fff69e6a5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:01.924357 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d8a98-5e61-47ef-a49a-b8fff69e6a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:01.924634 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d8a98-5e61-47ef-a49a-b8fff69e6a5c admin admin] PATH_INFO: `/v3/projects/68cdf36f2d8e4f0c8f8d5a5d6c81e538` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:01.925772 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55d8a98-5e61-47ef-a49a-b8fff69e6a5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=68cdf36f2d8e4f0c8f8d5a5d6c81e538)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:01.927890 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55d8a98-5e61-47ef-a49a-b8fff69e6a5c admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:01.975620 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d55d8a98-5e61-47ef-a49a-b8fff69e6a5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68cdf36f2d8e4f0c8f8d5a5d6c81e538', 'request_id': 'req-d55d8a98-5e61-47ef-a49a-b8fff69e6a5c'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:01.981661 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d55d8a98-5e61-47ef-a49a-b8fff69e6a5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68cdf36f2d8e4f0c8f8d5a5d6c81e538', 'request_id': 'req-d55d8a98-5e61-47ef-a49a-b8fff69e6a5c'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:01.988826 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d55d8a98-5e61-47ef-a49a-b8fff69e6a5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68cdf36f2d8e4f0c8f8d5a5d6c81e538', 'request_id': 'req-d55d8a98-5e61-47ef-a49a-b8fff69e6a5c'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:01.999283 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d55d8a98-5e61-47ef-a49a-b8fff69e6a5c admin admin] The token cache is being invalidate because project 68cdf36f2d8e4f0c8f8d5a5d6c81e538 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 08:46:01.999572 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d55d8a98-5e61-47ef-a49a-b8fff69e6a5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:02.000192 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d55d8a98-5e61-47ef-a49a-b8fff69e6a5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:02.001703 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 896/1778] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 08:46:01 2022] DELETE /identity/v3/projects/68cdf36f2d8e4f0c8f8d5a5d6c81e538 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:46:02.008395 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00437062-8ffe-4cb9-a78c-c7ff2c32780d None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:02.008888 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00437062-8ffe-4cb9-a78c-c7ff2c32780d None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:02.008980 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00437062-8ffe-4cb9-a78c-c7ff2c32780d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:02.029246 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-00437062-8ffe-4cb9-a78c-c7ff2c32780d None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:46:02.030479 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-00437062-8ffe-4cb9-a78c-c7ff2c32780d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:46:02.088333 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 883/1779] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:46:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:46:02.096189 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc98e58-a383-4755-aea3-e6026a45364e None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:02.096483 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc98e58-a383-4755-aea3-e6026a45364e None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:02.096770 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc98e58-a383-4755-aea3-e6026a45364e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:02.119292 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-dcc98e58-a383-4755-aea3-e6026a45364e None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:46:02.120776 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-dcc98e58-a383-4755-aea3-e6026a45364e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:46:02.129054 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 897/1780] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:46:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:46:02.133694 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00437062-8ffe-4cb9-a78c-c7ff2c32780d None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:02.143230 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00437062-8ffe-4cb9-a78c-c7ff2c32780d None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:02.148352 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e200bee-310a-4ee5-b0fb-0367f13c85d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:02.148957 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e200bee-310a-4ee5-b0fb-0367f13c85d2 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:02.149233 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e200bee-310a-4ee5-b0fb-0367f13c85d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:02.149502 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e200bee-310a-4ee5-b0fb-0367f13c85d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:02.150431 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e200bee-310a-4ee5-b0fb-0367f13c85d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:46:02.150736 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e200bee-310a-4ee5-b0fb-0367f13c85d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:02.152341 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e200bee-310a-4ee5-b0fb-0367f13c85d2 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:02.164176 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 884/1781] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 08:46:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:46:02.168392 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc98e58-a383-4755-aea3-e6026a45364e None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:02.183756 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc98e58-a383-4755-aea3-e6026a45364e None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:02.188754 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9ada65-0150-4cf6-835d-a0079df5ad85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:02.189346 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ada65-0150-4cf6-835d-a0079df5ad85 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:02.189662 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ada65-0150-4cf6-835d-a0079df5ad85 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:02.189962 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ada65-0150-4cf6-835d-a0079df5ad85 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:02.190602 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9ada65-0150-4cf6-835d-a0079df5ad85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:02.192518 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9ada65-0150-4cf6-835d-a0079df5ad85 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:02.219897 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 898/1782] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 08:46:02 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:46:02.223984 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e200bee-310a-4ee5-b0fb-0367f13c85d2 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:02.230273 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:46:02.234181 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e200bee-310a-4ee5-b0fb-0367f13c85d2 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:02.291267 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d74d0b1-739c-45a3-808e-d581c102d9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:02.291867 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d74d0b1-739c-45a3-808e-d581c102d9ae admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:02.292148 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d74d0b1-739c-45a3-808e-d581c102d9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:02.292425 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d74d0b1-739c-45a3-808e-d581c102d9ae admin admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:02.293064 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d74d0b1-739c-45a3-808e-d581c102d9ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:02.295025 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d74d0b1-739c-45a3-808e-d581c102d9ae admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:02.316262 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 885/1783] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:46:02 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 08:46:02.321297 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9ada65-0150-4cf6-835d-a0079df5ad85 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:02.332206 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9ada65-0150-4cf6-835d-a0079df5ad85 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:02.336899 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6027a973-ecf3-45b7-ab33-553fbea07b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:02.337522 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6027a973-ecf3-45b7-ab33-553fbea07b0a admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:02.337840 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6027a973-ecf3-45b7-ab33-553fbea07b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:02.338113 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6027a973-ecf3-45b7-ab33-553fbea07b0a admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:02.338749 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6027a973-ecf3-45b7-ab33-553fbea07b0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:02.340373 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6027a973-ecf3-45b7-ab33-553fbea07b0a admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:02.353775 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 899/1784] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:46:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 08:46:02.355032 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d74d0b1-739c-45a3-808e-d581c102d9ae admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:02.365500 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d74d0b1-739c-45a3-808e-d581c102d9ae admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:02.370574 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3804b2-0e56-40bb-b879-0fb0e88337e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:02.371177 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3804b2-0e56-40bb-b879-0fb0e88337e4 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:02.371474 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3804b2-0e56-40bb-b879-0fb0e88337e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:02.371752 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3804b2-0e56-40bb-b879-0fb0e88337e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:02.372390 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3804b2-0e56-40bb-b879-0fb0e88337e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:02.374030 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3804b2-0e56-40bb-b879-0fb0e88337e4 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:02.386847 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 886/1785] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:46:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:46:02.391157 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6027a973-ecf3-45b7-ab33-553fbea07b0a admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:02.401722 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6027a973-ecf3-45b7-ab33-553fbea07b0a admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:02.406551 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b75602-95b3-49c2-9c90-ea73da84a4cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:02.407206 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b75602-95b3-49c2-9c90-ea73da84a4cd admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:02.407490 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b75602-95b3-49c2-9c90-ea73da84a4cd admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:02.407764 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b75602-95b3-49c2-9c90-ea73da84a4cd admin admin] PATH_INFO: `/v3/projects/678aa241e39f4bea9fa9e73e82afbd0b/users/f05cbab4640048589b9509fd9e8f655a/roles/3248f0268738489b883041851d681bb7` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:02.422768 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b75602-95b3-49c2-9c90-ea73da84a4cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=678aa241e39f4bea9fa9e73e82afbd0b, user_id=f05cbab4640048589b9509fd9e8f655a, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:02.425397 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b75602-95b3-49c2-9c90-ea73da84a4cd admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:02.438705 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 900/1786] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:46:02 2022] PUT /identity/v3/projects/678aa241e39f4bea9fa9e73e82afbd0b/users/f05cbab4640048589b9509fd9e8f655a/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:46:02.443755 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb8387f-8476-437a-8af4-363e2a5dd4e1 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:02.444625 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb8387f-8476-437a-8af4-363e2a5dd4e1 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:02.444878 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb8387f-8476-437a-8af4-363e2a5dd4e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:02.466883 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-6fb8387f-8476-437a-8af4-363e2a5dd4e1 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:46:02.468100 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-6fb8387f-8476-437a-8af4-363e2a5dd4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:46:02.546583 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 887/1787] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:46:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:46:02.556921 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b75602-95b3-49c2-9c90-ea73da84a4cd admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:02.558654 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-18b75602-95b3-49c2-9c90-ea73da84a4cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:46:02.615169 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b75602-95b3-49c2-9c90-ea73da84a4cd admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:02.619996 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864c4ba4-df75-4cd0-8966-29b307be73c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:02.620612 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c4ba4-df75-4cd0-8966-29b307be73c7 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:02.620897 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c4ba4-df75-4cd0-8966-29b307be73c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:02.621173 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c4ba4-df75-4cd0-8966-29b307be73c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:02.632185 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864c4ba4-df75-4cd0-8966-29b307be73c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:02.633616 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864c4ba4-df75-4cd0-8966-29b307be73c7 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:02.638605 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:46:02.647924 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 901/1788] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:46:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:46:02.903201 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3355f3ac-919a-4c4f-a52e-1407c20a77c8 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:02.903488 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3355f3ac-919a-4c4f-a52e-1407c20a77c8 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:02.903791 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3355f3ac-919a-4c4f-a52e-1407c20a77c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:02.924742 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-3355f3ac-919a-4c4f-a52e-1407c20a77c8 None None] MFA Rules not processed for user `d716aed1f7684cda9b82ba0d5c3d24d4`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:46:02.925994 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-3355f3ac-919a-4c4f-a52e-1407c20a77c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:46:02.988701 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 888/1789] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:46:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:46:06.873451 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5667a13-dd73-4598-8aaa-5f13a8a7e67f None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:06.874034 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5667a13-dd73-4598-8aaa-5f13a8a7e67f None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:06.874134 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5667a13-dd73-4598-8aaa-5f13a8a7e67f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:06.902940 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-f5667a13-dd73-4598-8aaa-5f13a8a7e67f None None] MFA Rules not processed for user `f05cbab4640048589b9509fd9e8f655a`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:46:06.904162 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-f5667a13-dd73-4598-8aaa-5f13a8a7e67f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:46:06.976873 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 902/1790] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:46:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:46:06.981566 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3355f3ac-919a-4c4f-a52e-1407c20a77c8 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:06.995519 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3355f3ac-919a-4c4f-a52e-1407c20a77c8 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:07.000527 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25156f9c-83b9-4b62-98e1-20bb66971455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:07.001158 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25156f9c-83b9-4b62-98e1-20bb66971455 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:07.001443 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25156f9c-83b9-4b62-98e1-20bb66971455 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:07.001767 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25156f9c-83b9-4b62-98e1-20bb66971455 admin admin] PATH_INFO: `/v3/projects` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:07.002434 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25156f9c-83b9-4b62-98e1-20bb66971455 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:07.004366 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25156f9c-83b9-4b62-98e1-20bb66971455 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:07.042394 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 889/1791] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 08:46:06 2022] POST /identity/v3/projects => generated 385 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:46:07.044143 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5667a13-dd73-4598-8aaa-5f13a8a7e67f None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:07.059367 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5667a13-dd73-4598-8aaa-5f13a8a7e67f None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:07.120516 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed106d0-1883-4a54-ac9a-c7b7c98c25e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:07.121257 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed106d0-1883-4a54-ac9a-c7b7c98c25e3 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:07.121412 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed106d0-1883-4a54-ac9a-c7b7c98c25e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:07.121718 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed106d0-1883-4a54-ac9a-c7b7c98c25e3 admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:07.122979 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed106d0-1883-4a54-ac9a-c7b7c98c25e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:07.124287 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed106d0-1883-4a54-ac9a-c7b7c98c25e3 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:07.157828 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 903/1792] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:46:07 2022] POST /identity/v3/users => generated 299 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:46:07.166183 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25156f9c-83b9-4b62-98e1-20bb66971455 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:07.183637 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25156f9c-83b9-4b62-98e1-20bb66971455 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:07.188594 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d676e9-5abd-44ca-9ac3-4f46849fd1a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:07.189287 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d676e9-5abd-44ca-9ac3-4f46849fd1a7 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:07.189556 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d676e9-5abd-44ca-9ac3-4f46849fd1a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:07.189892 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d676e9-5abd-44ca-9ac3-4f46849fd1a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:07.190572 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d676e9-5abd-44ca-9ac3-4f46849fd1a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:07.192201 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d676e9-5abd-44ca-9ac3-4f46849fd1a7 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:07.193597 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:46:07.216170 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 890/1793] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:46:07 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:46:07.217025 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed106d0-1883-4a54-ac9a-c7b7c98c25e3 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:07.228140 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed106d0-1883-4a54-ac9a-c7b7c98c25e3 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:07.233085 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5551cd1e-4ef0-41e4-9beb-1f40ad3dcd66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:07.233796 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5551cd1e-4ef0-41e4-9beb-1f40ad3dcd66 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:07.234108 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5551cd1e-4ef0-41e4-9beb-1f40ad3dcd66 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:07.234386 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5551cd1e-4ef0-41e4-9beb-1f40ad3dcd66 admin admin] PATH_INFO: `/v3/projects/1df1d13607fb4c47970939aa8b1fd458/users/f6334153b91941b59ae8fa44e3073063/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:07.250698 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5551cd1e-4ef0-41e4-9beb-1f40ad3dcd66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1df1d13607fb4c47970939aa8b1fd458, user_id=f6334153b91941b59ae8fa44e3073063, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:07.253700 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5551cd1e-4ef0-41e4-9beb-1f40ad3dcd66 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:07.272503 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 904/1794] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:46:07 2022] PUT /identity/v3/projects/1df1d13607fb4c47970939aa8b1fd458/users/f6334153b91941b59ae8fa44e3073063/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 08:46:07.276054 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d676e9-5abd-44ca-9ac3-4f46849fd1a7 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:07.290841 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d676e9-5abd-44ca-9ac3-4f46849fd1a7 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:07.356844 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80884d6f-1398-456d-b9e6-be0acd2d7657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:07.357497 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80884d6f-1398-456d-b9e6-be0acd2d7657 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:07.357813 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80884d6f-1398-456d-b9e6-be0acd2d7657 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:07.358093 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80884d6f-1398-456d-b9e6-be0acd2d7657 admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:07.358749 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80884d6f-1398-456d-b9e6-be0acd2d7657 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:07.360378 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80884d6f-1398-456d-b9e6-be0acd2d7657 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:07.373970 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 891/1795] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:46:07 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:46:07.376377 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5551cd1e-4ef0-41e4-9beb-1f40ad3dcd66 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:07.387082 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5551cd1e-4ef0-41e4-9beb-1f40ad3dcd66 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:07.393012 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b92ce3-8468-464e-8ebc-cb3e73e5587d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:07.393240 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b92ce3-8468-464e-8ebc-cb3e73e5587d admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:07.393595 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b92ce3-8468-464e-8ebc-cb3e73e5587d admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:07.393982 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b92ce3-8468-464e-8ebc-cb3e73e5587d admin admin] PATH_INFO: `/v3/projects/1df1d13607fb4c47970939aa8b1fd458/users/f6334153b91941b59ae8fa44e3073063/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:07.396243 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b92ce3-8468-464e-8ebc-cb3e73e5587d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1df1d13607fb4c47970939aa8b1fd458, user_id=f6334153b91941b59ae8fa44e3073063, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:07.399602 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b92ce3-8468-464e-8ebc-cb3e73e5587d admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:07.411578 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 905/1796] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:46:07 2022] PUT /identity/v3/projects/1df1d13607fb4c47970939aa8b1fd458/users/f6334153b91941b59ae8fa44e3073063/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:46:11.638478 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80884d6f-1398-456d-b9e6-be0acd2d7657 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:11.641072 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-80884d6f-1398-456d-b9e6-be0acd2d7657 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:46:11.665489 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80884d6f-1398-456d-b9e6-be0acd2d7657 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:11.772272 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44943ab6-01e6-45af-bce6-99542c95d142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:11.772920 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44943ab6-01e6-45af-bce6-99542c95d142 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:11.773354 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44943ab6-01e6-45af-bce6-99542c95d142 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:11.774297 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44943ab6-01e6-45af-bce6-99542c95d142 admin admin] PATH_INFO: `/v3/users/e3713cd231374c4d93689823d3f1562b` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:11.775737 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44943ab6-01e6-45af-bce6-99542c95d142 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3713cd231374c4d93689823d3f1562b)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:11.777668 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44943ab6-01e6-45af-bce6-99542c95d142 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:11.916263 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-44943ab6-01e6-45af-bce6-99542c95d142 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3713cd231374c4d93689823d3f1562b'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:11.918264 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-44943ab6-01e6-45af-bce6-99542c95d142 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3713cd231374c4d93689823d3f1562b'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:11.946940 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-44943ab6-01e6-45af-bce6-99542c95d142 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3713cd231374c4d93689823d3f1562b'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:11.976853 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-44943ab6-01e6-45af-bce6-99542c95d142 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3713cd231374c4d93689823d3f1562b'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:11.977562 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-44943ab6-01e6-45af-bce6-99542c95d142 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3713cd231374c4d93689823d3f1562b'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:11.999444 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 892/1797] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 08:46:11 2022] DELETE /identity/v3/users/e3713cd231374c4d93689823d3f1562b => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 08:46:12.004507 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48c05b-f59c-463f-b0f1-cfd4a0f92a9d None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:12.004946 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48c05b-f59c-463f-b0f1-cfd4a0f92a9d None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:12.005106 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48c05b-f59c-463f-b0f1-cfd4a0f92a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:12.043433 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-dc48c05b-f59c-463f-b0f1-cfd4a0f92a9d None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:46:12.045122 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-dc48c05b-f59c-463f-b0f1-cfd4a0f92a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:46:12.100979 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:46:12.168891 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 906/1798] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:46:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:46:12.179024 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44943ab6-01e6-45af-bce6-99542c95d142 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:12.180536 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-44943ab6-01e6-45af-bce6-99542c95d142 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:46:12.207804 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:46:12.321585 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44943ab6-01e6-45af-bce6-99542c95d142 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:12.339072 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2598d067-5065-45dc-9dba-aa8f8710712b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:12.339738 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2598d067-5065-45dc-9dba-aa8f8710712b service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:12.340025 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2598d067-5065-45dc-9dba-aa8f8710712b service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:12.340306 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2598d067-5065-45dc-9dba-aa8f8710712b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:12.360227 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2598d067-5065-45dc-9dba-aa8f8710712b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:12.361725 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2598d067-5065-45dc-9dba-aa8f8710712b service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:12.405125 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 893/1799] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:46:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:46:12.780648 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38424fef-a88a-4ad9-ab4a-e7405380dbb2 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:12.780808 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38424fef-a88a-4ad9-ab4a-e7405380dbb2 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:12.781127 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38424fef-a88a-4ad9-ab4a-e7405380dbb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:12.822986 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-38424fef-a88a-4ad9-ab4a-e7405380dbb2 None None] MFA Rules not processed for user `f6334153b91941b59ae8fa44e3073063`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:46:12.824648 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-38424fef-a88a-4ad9-ab4a-e7405380dbb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:46:12.948322 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 907/1800] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:46:12 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:46:12.956528 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2598d067-5065-45dc-9dba-aa8f8710712b service neutron] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:12.957999 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-2598d067-5065-45dc-9dba-aa8f8710712b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:46:13.066731 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2598d067-5065-45dc-9dba-aa8f8710712b service neutron] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:13.070400 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3c5138-98ca-46ae-8127-4528782bfde4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71692d05fd324679b1598feb52b3fe6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:13.071275 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3c5138-98ca-46ae-8127-4528782bfde4 service glance] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:13.071275 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3c5138-98ca-46ae-8127-4528782bfde4 service glance] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:13.071536 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3c5138-98ca-46ae-8127-4528782bfde4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:13.073926 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-dd3c5138-98ca-46ae-8127-4528782bfde4 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:46:13.114717 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38424fef-a88a-4ad9-ab4a-e7405380dbb2 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:13.117167 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-38424fef-a88a-4ad9-ab4a-e7405380dbb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:46:13.140025 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38424fef-a88a-4ad9-ab4a-e7405380dbb2 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:13.146374 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bc9a79-70dc-4569-abb1-cbec851fa07c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:13.147215 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc9a79-70dc-4569-abb1-cbec851fa07c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:13.147500 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc9a79-70dc-4569-abb1-cbec851fa07c admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:13.147759 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc9a79-70dc-4569-abb1-cbec851fa07c admin admin] PATH_INFO: `/v3/projects/36ed3386378e4de49a4ede7466d1decd` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:13.148893 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bc9a79-70dc-4569-abb1-cbec851fa07c admin admin] RBAC: Authorizing `identity:delete_project(project_id=36ed3386378e4de49a4ede7466d1decd)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:13.151219 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bc9a79-70dc-4569-abb1-cbec851fa07c admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:13.154874 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3c5138-98ca-46ae-8127-4528782bfde4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:13.156415 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3c5138-98ca-46ae-8127-4528782bfde4 service glance] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:13.172007 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 894/1801] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:46:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:46:13.217445 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-09bc9a79-70dc-4569-abb1-cbec851fa07c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36ed3386378e4de49a4ede7466d1decd', 'request_id': 'req-09bc9a79-70dc-4569-abb1-cbec851fa07c'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:13.225510 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-09bc9a79-70dc-4569-abb1-cbec851fa07c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36ed3386378e4de49a4ede7466d1decd', 'request_id': 'req-09bc9a79-70dc-4569-abb1-cbec851fa07c'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:13.235412 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-09bc9a79-70dc-4569-abb1-cbec851fa07c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36ed3386378e4de49a4ede7466d1decd', 'request_id': 'req-09bc9a79-70dc-4569-abb1-cbec851fa07c'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:13.259137 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-09bc9a79-70dc-4569-abb1-cbec851fa07c admin admin] The token cache is being invalidate because project 36ed3386378e4de49a4ede7466d1decd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 08:46:13.259438 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-09bc9a79-70dc-4569-abb1-cbec851fa07c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:13.260121 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-09bc9a79-70dc-4569-abb1-cbec851fa07c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:13.261872 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 908/1802] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 08:46:13 2022] DELETE /identity/v3/projects/36ed3386378e4de49a4ede7466d1decd => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:46:13.266793 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3c5138-98ca-46ae-8127-4528782bfde4 service glance] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:13.289026 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3c5138-98ca-46ae-8127-4528782bfde4 service glance] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:13.333984 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e719d24-e8e6-4bfc-a2be-a6321b85f2b8 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:13.334274 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e719d24-e8e6-4bfc-a2be-a6321b85f2b8 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:13.334534 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e719d24-e8e6-4bfc-a2be-a6321b85f2b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:13.376382 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-3e719d24-e8e6-4bfc-a2be-a6321b85f2b8 None None] MFA Rules not processed for user `c5b2a0e623c348ad94d4820080677be6`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:46:13.377959 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-3e719d24-e8e6-4bfc-a2be-a6321b85f2b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:46:13.383383 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:46:13.416266 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7067f32-8935-4776-9751-be409ea319cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:13.416932 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7067f32-8935-4776-9751-be409ea319cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:13.417220 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7067f32-8935-4776-9751-be409ea319cc admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:13.417499 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7067f32-8935-4776-9751-be409ea319cc admin admin] PATH_INFO: `/v3/users/8d0f3b24ec7f4525a2a086d95dbd8395` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:13.419462 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7067f32-8935-4776-9751-be409ea319cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d0f3b24ec7f4525a2a086d95dbd8395)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:13.421440 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7067f32-8935-4776-9751-be409ea319cc admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:13.463031 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:46:13.489199 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 909/1803] 10.222.0.30 () {58 vars in 1009 bytes} [Sat Oct 15 08:46:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Oct 15 08:46:13.511356 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-b7067f32-8935-4776-9751-be409ea319cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d0f3b24ec7f4525a2a086d95dbd8395'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:13.512051 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-b7067f32-8935-4776-9751-be409ea319cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d0f3b24ec7f4525a2a086d95dbd8395'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:13.536598 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-b7067f32-8935-4776-9751-be409ea319cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d0f3b24ec7f4525a2a086d95dbd8395'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:13.548368 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-b7067f32-8935-4776-9751-be409ea319cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d0f3b24ec7f4525a2a086d95dbd8395'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:13.549493 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-b7067f32-8935-4776-9751-be409ea319cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d0f3b24ec7f4525a2a086d95dbd8395'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:13.554260 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e719d24-e8e6-4bfc-a2be-a6321b85f2b8 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:13.555987 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-3e719d24-e8e6-4bfc-a2be-a6321b85f2b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:46:13.568257 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 895/1804] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 08:46:13 2022] DELETE /identity/v3/users/8d0f3b24ec7f4525a2a086d95dbd8395 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:46:13.718493 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e719d24-e8e6-4bfc-a2be-a6321b85f2b8 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:13.842319 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da582a7-18a3-4100-a437-bc5b44fb2912 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e3009e20e4c49e5a74f7fd793b2c329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:13.843174 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da582a7-18a3-4100-a437-bc5b44fb2912 service swift] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:13.843461 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da582a7-18a3-4100-a437-bc5b44fb2912 service swift] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:13.843831 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da582a7-18a3-4100-a437-bc5b44fb2912 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:13.846614 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-5da582a7-18a3-4100-a437-bc5b44fb2912 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:46:13.962540 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da582a7-18a3-4100-a437-bc5b44fb2912 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:13.964309 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da582a7-18a3-4100-a437-bc5b44fb2912 service swift] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:13.975193 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 910/1805] 10.222.0.30 () {60 vars in 1384 bytes} [Sat Oct 15 08:46:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 15 08:46:14.095076 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7067f32-8935-4776-9751-be409ea319cc admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:14.095587 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-b7067f32-8935-4776-9751-be409ea319cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:46:14.255502 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7067f32-8935-4776-9751-be409ea319cc admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:14.265899 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68be3ed1-f1c9-497b-a7a3-30aeb5fb17c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:14.266030 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68be3ed1-f1c9-497b-a7a3-30aeb5fb17c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:14.267037 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68be3ed1-f1c9-497b-a7a3-30aeb5fb17c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:14.267037 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68be3ed1-f1c9-497b-a7a3-30aeb5fb17c3 admin admin] PATH_INFO: `/v3/projects/5450acaf9fc94ca0baad9a74c42b3652` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:14.269884 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68be3ed1-f1c9-497b-a7a3-30aeb5fb17c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5450acaf9fc94ca0baad9a74c42b3652)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:14.272236 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68be3ed1-f1c9-497b-a7a3-30aeb5fb17c3 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:14.398625 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-68be3ed1-f1c9-497b-a7a3-30aeb5fb17c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5450acaf9fc94ca0baad9a74c42b3652', 'request_id': 'req-68be3ed1-f1c9-497b-a7a3-30aeb5fb17c3'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:14.419872 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-68be3ed1-f1c9-497b-a7a3-30aeb5fb17c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5450acaf9fc94ca0baad9a74c42b3652', 'request_id': 'req-68be3ed1-f1c9-497b-a7a3-30aeb5fb17c3'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:14.464096 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-68be3ed1-f1c9-497b-a7a3-30aeb5fb17c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5450acaf9fc94ca0baad9a74c42b3652', 'request_id': 'req-68be3ed1-f1c9-497b-a7a3-30aeb5fb17c3'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:14.502174 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:46:14.509717 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-68be3ed1-f1c9-497b-a7a3-30aeb5fb17c3 admin admin] The token cache is being invalidate because project 5450acaf9fc94ca0baad9a74c42b3652 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 08:46:14.510150 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-68be3ed1-f1c9-497b-a7a3-30aeb5fb17c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:14.513738 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-68be3ed1-f1c9-497b-a7a3-30aeb5fb17c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:14.517285 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 896/1806] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 08:46:14 2022] DELETE /identity/v3/projects/5450acaf9fc94ca0baad9a74c42b3652 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:46:14.521955 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5655b912-562d-444d-8631-645f8b2ededf None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:14.522491 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5655b912-562d-444d-8631-645f8b2ededf None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:14.522580 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5655b912-562d-444d-8631-645f8b2ededf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:14.562995 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-5655b912-562d-444d-8631-645f8b2ededf None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:46:14.564419 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-5655b912-562d-444d-8631-645f8b2ededf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:46:14.606778 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:46:14.655509 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 911/1807] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:46:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:46:14.663723 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c317aa-0a6e-4da9-abc8-b8fc6aa02910 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:14.664020 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c317aa-0a6e-4da9-abc8-b8fc6aa02910 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:14.664291 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c317aa-0a6e-4da9-abc8-b8fc6aa02910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:14.693551 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-39c317aa-0a6e-4da9-abc8-b8fc6aa02910 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:46:14.697834 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-39c317aa-0a6e-4da9-abc8-b8fc6aa02910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:46:14.707235 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 897/1808] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:46:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:46:14.711172 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5655b912-562d-444d-8631-645f8b2ededf None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:14.728199 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5655b912-562d-444d-8631-645f8b2ededf None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:14.756882 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26742d09-8a02-48e5-8a3b-60c15dbd7dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:14.759243 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26742d09-8a02-48e5-8a3b-60c15dbd7dc2 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:14.759799 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26742d09-8a02-48e5-8a3b-60c15dbd7dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:14.760886 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26742d09-8a02-48e5-8a3b-60c15dbd7dc2 admin admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:14.761771 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26742d09-8a02-48e5-8a3b-60c15dbd7dc2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:46:14.764172 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26742d09-8a02-48e5-8a3b-60c15dbd7dc2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:14.766989 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26742d09-8a02-48e5-8a3b-60c15dbd7dc2 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:14.796283 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 912/1809] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 08:46:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:46:14.800530 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c317aa-0a6e-4da9-abc8-b8fc6aa02910 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:14.825866 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c317aa-0a6e-4da9-abc8-b8fc6aa02910 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:14.832496 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d628036-1e91-499a-8f40-57bfe9160ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:14.833188 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d628036-1e91-499a-8f40-57bfe9160ae1 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:14.833534 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d628036-1e91-499a-8f40-57bfe9160ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:14.833906 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d628036-1e91-499a-8f40-57bfe9160ae1 admin admin] PATH_INFO: `/v3/projects` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:14.834666 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d628036-1e91-499a-8f40-57bfe9160ae1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:14.836708 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d628036-1e91-499a-8f40-57bfe9160ae1 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:14.890244 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 898/1810] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 08:46:14 2022] POST /identity/v3/projects => generated 369 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:46:14.891134 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26742d09-8a02-48e5-8a3b-60c15dbd7dc2 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:14.908901 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26742d09-8a02-48e5-8a3b-60c15dbd7dc2 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:15.041951 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba39300-52fb-489a-a2a1-f2817aa17f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:15.042646 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba39300-52fb-489a-a2a1-f2817aa17f56 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:15.042939 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba39300-52fb-489a-a2a1-f2817aa17f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:15.043218 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba39300-52fb-489a-a2a1-f2817aa17f56 admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:15.043868 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba39300-52fb-489a-a2a1-f2817aa17f56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:15.045858 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba39300-52fb-489a-a2a1-f2817aa17f56 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:15.092060 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 913/1811] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:46:14 2022] POST /identity/v3/users => generated 285 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:46:15.098683 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d628036-1e91-499a-8f40-57bfe9160ae1 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:15.115101 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d628036-1e91-499a-8f40-57bfe9160ae1 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:15.129984 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0175ccf2-78b7-4a58-8a41-c5d74ae0758f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:15.130539 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0175ccf2-78b7-4a58-8a41-c5d74ae0758f admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:15.131009 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0175ccf2-78b7-4a58-8a41-c5d74ae0758f admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:15.131178 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0175ccf2-78b7-4a58-8a41-c5d74ae0758f admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:15.131818 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0175ccf2-78b7-4a58-8a41-c5d74ae0758f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:15.133616 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0175ccf2-78b7-4a58-8a41-c5d74ae0758f admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:15.159936 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 899/1812] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:46:15 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:46:15.160980 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba39300-52fb-489a-a2a1-f2817aa17f56 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:15.174821 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba39300-52fb-489a-a2a1-f2817aa17f56 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:15.183048 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed0b7e6-c548-4c7f-9665-6bdd43ffa2b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:15.183955 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed0b7e6-c548-4c7f-9665-6bdd43ffa2b2 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:15.184573 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed0b7e6-c548-4c7f-9665-6bdd43ffa2b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:15.185062 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed0b7e6-c548-4c7f-9665-6bdd43ffa2b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:15.185981 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed0b7e6-c548-4c7f-9665-6bdd43ffa2b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:15.187869 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed0b7e6-c548-4c7f-9665-6bdd43ffa2b2 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:15.210736 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 914/1813] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:46:15 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 08:46:15.215003 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0175ccf2-78b7-4a58-8a41-c5d74ae0758f admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:15.233823 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0175ccf2-78b7-4a58-8a41-c5d74ae0758f admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:15.239230 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5736e60-381c-40e3-8d31-2dc771f35226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:15.240333 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5736e60-381c-40e3-8d31-2dc771f35226 admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:15.240723 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5736e60-381c-40e3-8d31-2dc771f35226 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:15.241081 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5736e60-381c-40e3-8d31-2dc771f35226 admin admin] PATH_INFO: `/v3/projects/da6bfc85984845568158e78994bd5caa/users/aab1effeb98a450b8480691826fb92ad/roles/3248f0268738489b883041851d681bb7` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:15.261530 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5736e60-381c-40e3-8d31-2dc771f35226 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da6bfc85984845568158e78994bd5caa, user_id=aab1effeb98a450b8480691826fb92ad, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:15.270202 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5736e60-381c-40e3-8d31-2dc771f35226 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:15.337052 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 900/1814] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:46:15 2022] PUT /identity/v3/projects/da6bfc85984845568158e78994bd5caa/users/aab1effeb98a450b8480691826fb92ad/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 08:46:15.338664 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f997a-c4cb-4730-85b1-0f1f53e50a64 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:15.339000 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f997a-c4cb-4730-85b1-0f1f53e50a64 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:15.339289 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f997a-c4cb-4730-85b1-0f1f53e50a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:15.432336 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-016f997a-c4cb-4730-85b1-0f1f53e50a64 None None] MFA Rules not processed for user `aab1effeb98a450b8480691826fb92ad`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:46:15.434102 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-016f997a-c4cb-4730-85b1-0f1f53e50a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:46:15.543794 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 915/1815] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:46:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:46:15.549092 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5736e60-381c-40e3-8d31-2dc771f35226 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:15.648222 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5736e60-381c-40e3-8d31-2dc771f35226 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:15.658110 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f193b-067e-478b-b2e0-d24fb99c8436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:15.659505 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f193b-067e-478b-b2e0-d24fb99c8436 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:15.659909 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f193b-067e-478b-b2e0-d24fb99c8436 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:15.660266 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f193b-067e-478b-b2e0-d24fb99c8436 admin admin] PATH_INFO: `/v3/projects` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:15.661186 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6f193b-067e-478b-b2e0-d24fb99c8436 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:15.664111 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6f193b-067e-478b-b2e0-d24fb99c8436 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:15.716261 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 901/1816] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 08:46:15 2022] POST /identity/v3/projects => generated 369 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:46:15.720406 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016f997a-c4cb-4730-85b1-0f1f53e50a64 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:15.734990 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016f997a-c4cb-4730-85b1-0f1f53e50a64 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:15.781757 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:46:15.846802 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b24194f-45d7-434b-8346-19f9c79b14b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:15.847892 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b24194f-45d7-434b-8346-19f9c79b14b0 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:15.848783 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b24194f-45d7-434b-8346-19f9c79b14b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:15.849218 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b24194f-45d7-434b-8346-19f9c79b14b0 admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:15.850717 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b24194f-45d7-434b-8346-19f9c79b14b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:15.854869 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b24194f-45d7-434b-8346-19f9c79b14b0 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:15.919935 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 916/1817] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:46:15 2022] POST /identity/v3/users => generated 291 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 08:46:15.922036 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f193b-067e-478b-b2e0-d24fb99c8436 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:15.940266 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f193b-067e-478b-b2e0-d24fb99c8436 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:15.947630 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71570f3-b37e-4c66-beb2-29891f64dc1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:15.948404 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71570f3-b37e-4c66-beb2-29891f64dc1a admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:15.948789 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71570f3-b37e-4c66-beb2-29891f64dc1a admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:15.949427 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71570f3-b37e-4c66-beb2-29891f64dc1a admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:15.954116 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71570f3-b37e-4c66-beb2-29891f64dc1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:15.956512 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71570f3-b37e-4c66-beb2-29891f64dc1a admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:15.983526 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 902/1818] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:46:15 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:46:15.983819 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b24194f-45d7-434b-8346-19f9c79b14b0 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:16.003614 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b24194f-45d7-434b-8346-19f9c79b14b0 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:16.015158 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33eee0d-0140-4227-8208-00e17143a21b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:16.016310 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33eee0d-0140-4227-8208-00e17143a21b admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:16.016817 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33eee0d-0140-4227-8208-00e17143a21b admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:16.017326 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33eee0d-0140-4227-8208-00e17143a21b admin admin] PATH_INFO: `/v3/projects/ce9ef32a0a5b4be8b2ed14d41fea9c85/users/12d4341988514da5ba8fd6b53d01cab7/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:16.042428 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33eee0d-0140-4227-8208-00e17143a21b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce9ef32a0a5b4be8b2ed14d41fea9c85, user_id=12d4341988514da5ba8fd6b53d01cab7, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:16.047475 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33eee0d-0140-4227-8208-00e17143a21b admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:16.079708 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 917/1819] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:46:15 2022] PUT /identity/v3/projects/ce9ef32a0a5b4be8b2ed14d41fea9c85/users/12d4341988514da5ba8fd6b53d01cab7/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:46:16.085622 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71570f3-b37e-4c66-beb2-29891f64dc1a admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:16.101912 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71570f3-b37e-4c66-beb2-29891f64dc1a admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:16.204373 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76a2501-aaf4-4f6e-a41f-3da9821d43ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:16.205191 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a2501-aaf4-4f6e-a41f-3da9821d43ef admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:16.205573 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a2501-aaf4-4f6e-a41f-3da9821d43ef admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:16.205985 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a2501-aaf4-4f6e-a41f-3da9821d43ef admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:16.206715 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76a2501-aaf4-4f6e-a41f-3da9821d43ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:16.208401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76a2501-aaf4-4f6e-a41f-3da9821d43ef admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:16.220037 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 903/1820] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:46:16 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:46:16.224584 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33eee0d-0140-4227-8208-00e17143a21b admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:16.239785 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33eee0d-0140-4227-8208-00e17143a21b admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:16.245133 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d36c30f-8061-4c57-b970-405ded69e9bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:16.245847 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36c30f-8061-4c57-b970-405ded69e9bc admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:16.246129 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36c30f-8061-4c57-b970-405ded69e9bc admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:16.246406 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36c30f-8061-4c57-b970-405ded69e9bc admin admin] PATH_INFO: `/v3/projects/ce9ef32a0a5b4be8b2ed14d41fea9c85/users/12d4341988514da5ba8fd6b53d01cab7/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:16.248315 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d36c30f-8061-4c57-b970-405ded69e9bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce9ef32a0a5b4be8b2ed14d41fea9c85, user_id=12d4341988514da5ba8fd6b53d01cab7, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:16.251329 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d36c30f-8061-4c57-b970-405ded69e9bc admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:16.265384 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 918/1821] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:46:16 2022] PUT /identity/v3/projects/ce9ef32a0a5b4be8b2ed14d41fea9c85/users/12d4341988514da5ba8fd6b53d01cab7/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:46:16.278204 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0de0d5-713d-4914-aa6d-d513e4b48bd4 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:16.278530 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0de0d5-713d-4914-aa6d-d513e4b48bd4 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:16.278811 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0de0d5-713d-4914-aa6d-d513e4b48bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:16.289582 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:46:16.333648 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-0d0de0d5-713d-4914-aa6d-d513e4b48bd4 None None] MFA Rules not processed for user `12d4341988514da5ba8fd6b53d01cab7`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:46:16.335641 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-0d0de0d5-713d-4914-aa6d-d513e4b48bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:46:16.430070 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 904/1822] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:46:16 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:46:16.443333 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d36c30f-8061-4c57-b970-405ded69e9bc admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:16.446152 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-7d36c30f-8061-4c57-b970-405ded69e9bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:46:16.534287 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d36c30f-8061-4c57-b970-405ded69e9bc admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:16.539389 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8468ec8d-2063-4c28-8b5e-c221f50e113e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:16.540218 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8468ec8d-2063-4c28-8b5e-c221f50e113e service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:16.540655 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8468ec8d-2063-4c28-8b5e-c221f50e113e service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:16.541105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8468ec8d-2063-4c28-8b5e-c221f50e113e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:16.552333 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8468ec8d-2063-4c28-8b5e-c221f50e113e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:16.553838 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8468ec8d-2063-4c28-8b5e-c221f50e113e service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:16.564410 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 919/1823] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:46:16 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:46:16.647992 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0de0d5-713d-4914-aa6d-d513e4b48bd4 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:16.650371 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-0d0de0d5-713d-4914-aa6d-d513e4b48bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:46:16.678125 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0de0d5-713d-4914-aa6d-d513e4b48bd4 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:16.699329 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82a162b-b951-4d7e-97f5-e4895510fdf6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:16.699979 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a162b-b951-4d7e-97f5-e4895510fdf6 service cinder] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:16.700266 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a162b-b951-4d7e-97f5-e4895510fdf6 service cinder] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:16.700546 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a162b-b951-4d7e-97f5-e4895510fdf6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:16.818052 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82a162b-b951-4d7e-97f5-e4895510fdf6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:16.828411 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82a162b-b951-4d7e-97f5-e4895510fdf6 service cinder] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:16.862785 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 905/1824] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:46:16 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:46:21.082596 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8468ec8d-2063-4c28-8b5e-c221f50e113e service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:21.171221 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8468ec8d-2063-4c28-8b5e-c221f50e113e service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:21.179347 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8400b2f-e4e3-4394-9087-636e1d8a93b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:21.180010 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8400b2f-e4e3-4394-9087-636e1d8a93b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:21.180289 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8400b2f-e4e3-4394-9087-636e1d8a93b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:21.180562 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8400b2f-e4e3-4394-9087-636e1d8a93b9 admin admin] PATH_INFO: `/v3/users/aab1effeb98a450b8480691826fb92ad` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:21.181740 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8400b2f-e4e3-4394-9087-636e1d8a93b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aab1effeb98a450b8480691826fb92ad)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:21.187886 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8400b2f-e4e3-4394-9087-636e1d8a93b9 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:21.236781 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:46:21.258115 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-a8400b2f-e4e3-4394-9087-636e1d8a93b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aab1effeb98a450b8480691826fb92ad'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:21.258737 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-a8400b2f-e4e3-4394-9087-636e1d8a93b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aab1effeb98a450b8480691826fb92ad'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:21.266209 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-a8400b2f-e4e3-4394-9087-636e1d8a93b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aab1effeb98a450b8480691826fb92ad'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:21.291306 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-a8400b2f-e4e3-4394-9087-636e1d8a93b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aab1effeb98a450b8480691826fb92ad'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:21.298943 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-a8400b2f-e4e3-4394-9087-636e1d8a93b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aab1effeb98a450b8480691826fb92ad'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:21.305712 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 920/1825] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 08:46:21 2022] DELETE /identity/v3/users/aab1effeb98a450b8480691826fb92ad => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:46:21.310379 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60685683-585f-4549-80fd-0a1568e5de23 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:21.310949 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60685683-585f-4549-80fd-0a1568e5de23 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:21.311186 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60685683-585f-4549-80fd-0a1568e5de23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:21.332002 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-60685683-585f-4549-80fd-0a1568e5de23 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:46:21.340535 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-60685683-585f-4549-80fd-0a1568e5de23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:46:21.430540 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 906/1826] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:46:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 08:46:21.441080 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8400b2f-e4e3-4394-9087-636e1d8a93b9 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:21.441711 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-a8400b2f-e4e3-4394-9087-636e1d8a93b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:46:21.514825 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8400b2f-e4e3-4394-9087-636e1d8a93b9 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:21.520615 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02242b4c-7a61-4cd1-b58e-59995bfc7a82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:21.521262 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02242b4c-7a61-4cd1-b58e-59995bfc7a82 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:21.521560 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02242b4c-7a61-4cd1-b58e-59995bfc7a82 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:21.522562 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02242b4c-7a61-4cd1-b58e-59995bfc7a82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:21.534493 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02242b4c-7a61-4cd1-b58e-59995bfc7a82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:21.538140 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02242b4c-7a61-4cd1-b58e-59995bfc7a82 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:21.550242 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 921/1827] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:46:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 08:46:21.919749 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60685683-585f-4549-80fd-0a1568e5de23 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:21.921424 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-60685683-585f-4549-80fd-0a1568e5de23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:46:21.939347 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60685683-585f-4549-80fd-0a1568e5de23 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:21.944258 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7522c4-fe64-44af-aee1-bf403bd7b18f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:21.944909 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7522c4-fe64-44af-aee1-bf403bd7b18f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:21.945189 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7522c4-fe64-44af-aee1-bf403bd7b18f admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:21.945472 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7522c4-fe64-44af-aee1-bf403bd7b18f admin admin] PATH_INFO: `/v3/projects/da6bfc85984845568158e78994bd5caa` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:21.946640 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7522c4-fe64-44af-aee1-bf403bd7b18f admin admin] RBAC: Authorizing `identity:delete_project(project_id=da6bfc85984845568158e78994bd5caa)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:21.948620 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7522c4-fe64-44af-aee1-bf403bd7b18f admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:21.977432 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:46:22.007498 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-ab7522c4-fe64-44af-aee1-bf403bd7b18f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da6bfc85984845568158e78994bd5caa', 'request_id': 'req-ab7522c4-fe64-44af-aee1-bf403bd7b18f'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:22.015440 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-ab7522c4-fe64-44af-aee1-bf403bd7b18f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da6bfc85984845568158e78994bd5caa', 'request_id': 'req-ab7522c4-fe64-44af-aee1-bf403bd7b18f'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:22.025376 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-ab7522c4-fe64-44af-aee1-bf403bd7b18f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da6bfc85984845568158e78994bd5caa', 'request_id': 'req-ab7522c4-fe64-44af-aee1-bf403bd7b18f'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:22.041902 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-ab7522c4-fe64-44af-aee1-bf403bd7b18f admin admin] The token cache is being invalidate because project da6bfc85984845568158e78994bd5caa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 08:46:22.042195 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-ab7522c4-fe64-44af-aee1-bf403bd7b18f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:22.042848 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-ab7522c4-fe64-44af-aee1-bf403bd7b18f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:22.044938 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 907/1828] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 08:46:21 2022] DELETE /identity/v3/projects/da6bfc85984845568158e78994bd5caa => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:46:22.048880 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02242b4c-7a61-4cd1-b58e-59995bfc7a82 service neutron] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:22.059813 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02242b4c-7a61-4cd1-b58e-59995bfc7a82 service neutron] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:22.137613 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae913dc-8c99-4bc5-9f0c-9dd24ca4fba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:22.138432 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae913dc-8c99-4bc5-9f0c-9dd24ca4fba8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:22.138752 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae913dc-8c99-4bc5-9f0c-9dd24ca4fba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:22.139095 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae913dc-8c99-4bc5-9f0c-9dd24ca4fba8 admin admin] PATH_INFO: `/v3/users/12d4341988514da5ba8fd6b53d01cab7` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:22.140292 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae913dc-8c99-4bc5-9f0c-9dd24ca4fba8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12d4341988514da5ba8fd6b53d01cab7)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:22.142259 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae913dc-8c99-4bc5-9f0c-9dd24ca4fba8 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:22.196704 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-9ae913dc-8c99-4bc5-9f0c-9dd24ca4fba8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12d4341988514da5ba8fd6b53d01cab7'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:22.197750 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-9ae913dc-8c99-4bc5-9f0c-9dd24ca4fba8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12d4341988514da5ba8fd6b53d01cab7'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:22.203468 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:46:22.210089 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-9ae913dc-8c99-4bc5-9f0c-9dd24ca4fba8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12d4341988514da5ba8fd6b53d01cab7'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:22.226089 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-9ae913dc-8c99-4bc5-9f0c-9dd24ca4fba8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12d4341988514da5ba8fd6b53d01cab7'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:22.235477 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-9ae913dc-8c99-4bc5-9f0c-9dd24ca4fba8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12d4341988514da5ba8fd6b53d01cab7'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:22.243956 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 922/1829] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 08:46:22 2022] DELETE /identity/v3/users/12d4341988514da5ba8fd6b53d01cab7 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 08:46:22.607237 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7522c4-fe64-44af-aee1-bf403bd7b18f admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:22.608773 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-ab7522c4-fe64-44af-aee1-bf403bd7b18f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:46:22.624483 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7522c4-fe64-44af-aee1-bf403bd7b18f admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:22.700286 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644e5f2a-1940-4830-af1f-2148a665d003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:22.700961 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e5f2a-1940-4830-af1f-2148a665d003 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:22.701241 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e5f2a-1940-4830-af1f-2148a665d003 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:22.701514 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e5f2a-1940-4830-af1f-2148a665d003 admin admin] PATH_INFO: `/v3/projects/ce9ef32a0a5b4be8b2ed14d41fea9c85` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:22.702709 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644e5f2a-1940-4830-af1f-2148a665d003 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce9ef32a0a5b4be8b2ed14d41fea9c85)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:22.704889 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644e5f2a-1940-4830-af1f-2148a665d003 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:22.767902 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-644e5f2a-1940-4830-af1f-2148a665d003 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce9ef32a0a5b4be8b2ed14d41fea9c85', 'request_id': 'req-644e5f2a-1940-4830-af1f-2148a665d003'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:22.775557 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-644e5f2a-1940-4830-af1f-2148a665d003 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce9ef32a0a5b4be8b2ed14d41fea9c85', 'request_id': 'req-644e5f2a-1940-4830-af1f-2148a665d003'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:22.785805 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-644e5f2a-1940-4830-af1f-2148a665d003 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce9ef32a0a5b4be8b2ed14d41fea9c85', 'request_id': 'req-644e5f2a-1940-4830-af1f-2148a665d003'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:22.805053 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-644e5f2a-1940-4830-af1f-2148a665d003 admin admin] The token cache is being invalidate because project ce9ef32a0a5b4be8b2ed14d41fea9c85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 08:46:22.805351 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-644e5f2a-1940-4830-af1f-2148a665d003 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:22.806146 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-644e5f2a-1940-4830-af1f-2148a665d003 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:22.808274 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 908/1830] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 08:46:22 2022] DELETE /identity/v3/projects/ce9ef32a0a5b4be8b2ed14d41fea9c85 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:46:22.814421 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a4a625-729b-4333-af8c-114f6376ecfb None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:22.814754 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a4a625-729b-4333-af8c-114f6376ecfb None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:22.815034 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a4a625-729b-4333-af8c-114f6376ecfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:22.842068 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-f0a4a625-729b-4333-af8c-114f6376ecfb None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:46:22.843435 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-f0a4a625-729b-4333-af8c-114f6376ecfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:46:22.938446 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 923/1831] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:46:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 08:46:22.948805 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6580293e-962b-45c4-9568-541a5a228c1d None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:22.949108 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6580293e-962b-45c4-9568-541a5a228c1d None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:22.949386 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6580293e-962b-45c4-9568-541a5a228c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:22.972268 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-6580293e-962b-45c4-9568-541a5a228c1d None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:46:22.972993 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-6580293e-962b-45c4-9568-541a5a228c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:46:22.988239 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a4a625-729b-4333-af8c-114f6376ecfb None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:22.995667 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 909/1832] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:46:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:46:22.999801 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a4a625-729b-4333-af8c-114f6376ecfb None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:23.004780 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765a6d4f-efaa-4802-a1a2-00cb77c951be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:23.005454 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765a6d4f-efaa-4802-a1a2-00cb77c951be admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:23.005766 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765a6d4f-efaa-4802-a1a2-00cb77c951be admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:23.006077 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765a6d4f-efaa-4802-a1a2-00cb77c951be admin admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:23.006731 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765a6d4f-efaa-4802-a1a2-00cb77c951be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:46:23.007134 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765a6d4f-efaa-4802-a1a2-00cb77c951be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:23.009200 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765a6d4f-efaa-4802-a1a2-00cb77c951be admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:23.029584 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6580293e-962b-45c4-9568-541a5a228c1d None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:23.035576 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 924/1833] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 08:46:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:46:23.039711 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6580293e-962b-45c4-9568-541a5a228c1d None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:23.047214 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8952276f-039e-4455-9b81-e080efc2eba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:23.048221 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8952276f-039e-4455-9b81-e080efc2eba8 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:23.048716 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8952276f-039e-4455-9b81-e080efc2eba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:23.049097 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8952276f-039e-4455-9b81-e080efc2eba8 admin admin] PATH_INFO: `/v3/projects` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:23.049899 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8952276f-039e-4455-9b81-e080efc2eba8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:23.051973 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8952276f-039e-4455-9b81-e080efc2eba8 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:23.093622 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 910/1834] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 08:46:23 2022] POST /identity/v3/projects => generated 385 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:46:23.097855 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765a6d4f-efaa-4802-a1a2-00cb77c951be admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:23.109754 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765a6d4f-efaa-4802-a1a2-00cb77c951be admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:23.182405 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461f9c78-4537-4c5e-9500-69fefec8467c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:23.183097 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f9c78-4537-4c5e-9500-69fefec8467c admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:23.183453 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f9c78-4537-4c5e-9500-69fefec8467c admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:23.183792 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f9c78-4537-4c5e-9500-69fefec8467c admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:23.184516 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461f9c78-4537-4c5e-9500-69fefec8467c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:23.186547 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461f9c78-4537-4c5e-9500-69fefec8467c admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:23.237796 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 925/1835] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:46:23 2022] POST /identity/v3/users => generated 293 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 08:46:23.238116 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8952276f-039e-4455-9b81-e080efc2eba8 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:23.262257 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8952276f-039e-4455-9b81-e080efc2eba8 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:23.276710 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55559d35-d238-4ae3-b122-51dd2d105857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:23.276710 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55559d35-d238-4ae3-b122-51dd2d105857 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:23.277018 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55559d35-d238-4ae3-b122-51dd2d105857 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:23.277280 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55559d35-d238-4ae3-b122-51dd2d105857 admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:23.280131 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55559d35-d238-4ae3-b122-51dd2d105857 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:23.281535 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55559d35-d238-4ae3-b122-51dd2d105857 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:23.310303 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 911/1836] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:46:23 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:46:23.314564 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461f9c78-4537-4c5e-9500-69fefec8467c admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:23.371685 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461f9c78-4537-4c5e-9500-69fefec8467c admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:23.389012 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e86b6c-ea58-410a-a662-41ff0856c840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:23.389810 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e86b6c-ea58-410a-a662-41ff0856c840 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:23.390156 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e86b6c-ea58-410a-a662-41ff0856c840 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:23.390502 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e86b6c-ea58-410a-a662-41ff0856c840 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:23.391261 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e86b6c-ea58-410a-a662-41ff0856c840 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:23.393000 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e86b6c-ea58-410a-a662-41ff0856c840 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:23.410642 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 926/1837] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:46:23 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:46:23.414972 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55559d35-d238-4ae3-b122-51dd2d105857 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:23.420344 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:46:23.432628 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55559d35-d238-4ae3-b122-51dd2d105857 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:23.441306 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110226a5-691d-45ad-b897-738f138cab4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:23.442177 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110226a5-691d-45ad-b897-738f138cab4a admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:23.442525 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110226a5-691d-45ad-b897-738f138cab4a admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:23.442994 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110226a5-691d-45ad-b897-738f138cab4a admin admin] PATH_INFO: `/v3/projects/c14e5bb2fe8b46ca8cad7b981b042916/users/1124027bf7264c0396c43e4ba435758a/roles/3248f0268738489b883041851d681bb7` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:23.463056 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110226a5-691d-45ad-b897-738f138cab4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c14e5bb2fe8b46ca8cad7b981b042916, user_id=1124027bf7264c0396c43e4ba435758a, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:23.466017 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110226a5-691d-45ad-b897-738f138cab4a admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:23.483047 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 912/1838] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:46:23 2022] PUT /identity/v3/projects/c14e5bb2fe8b46ca8cad7b981b042916/users/1124027bf7264c0396c43e4ba435758a/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:46:23.495440 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f487411-ea8c-4ce5-8042-2e59f67d7f71 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:23.495786 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f487411-ea8c-4ce5-8042-2e59f67d7f71 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:23.496087 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f487411-ea8c-4ce5-8042-2e59f67d7f71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:23.533757 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-8f487411-ea8c-4ce5-8042-2e59f67d7f71 None None] MFA Rules not processed for user `1124027bf7264c0396c43e4ba435758a`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:46:23.535030 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-8f487411-ea8c-4ce5-8042-2e59f67d7f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:46:23.581168 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:46:23.629166 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 927/1839] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:46:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:46:23.633895 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110226a5-691d-45ad-b897-738f138cab4a admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:23.711125 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110226a5-691d-45ad-b897-738f138cab4a admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:23.717893 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e518a2-2dc5-448b-b466-0051e9b3ea1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:23.718523 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e518a2-2dc5-448b-b466-0051e9b3ea1b admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:23.718808 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e518a2-2dc5-448b-b466-0051e9b3ea1b admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:23.719084 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e518a2-2dc5-448b-b466-0051e9b3ea1b admin admin] PATH_INFO: `/v3/projects` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:23.719728 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e518a2-2dc5-448b-b466-0051e9b3ea1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:23.721650 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e518a2-2dc5-448b-b466-0051e9b3ea1b admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:23.770596 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 913/1840] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 08:46:23 2022] POST /identity/v3/projects => generated 385 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:46:23.774255 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f487411-ea8c-4ce5-8042-2e59f67d7f71 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:23.799199 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f487411-ea8c-4ce5-8042-2e59f67d7f71 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:23.895845 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3d4756-cb34-4ea8-83ae-48df226dab10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:23.898905 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d4756-cb34-4ea8-83ae-48df226dab10 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:23.899189 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d4756-cb34-4ea8-83ae-48df226dab10 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:23.899466 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d4756-cb34-4ea8-83ae-48df226dab10 admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:23.900386 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3d4756-cb34-4ea8-83ae-48df226dab10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:23.902401 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3d4756-cb34-4ea8-83ae-48df226dab10 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:23.938593 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 928/1841] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:46:23 2022] POST /identity/v3/users => generated 293 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:46:23.947500 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e518a2-2dc5-448b-b466-0051e9b3ea1b admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:23.964773 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e518a2-2dc5-448b-b466-0051e9b3ea1b admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:23.971726 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6b5036-9671-4d1f-9794-ba25679d0669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:23.972602 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b5036-9671-4d1f-9794-ba25679d0669 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:23.972882 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b5036-9671-4d1f-9794-ba25679d0669 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:23.973157 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b5036-9671-4d1f-9794-ba25679d0669 admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:23.973850 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6b5036-9671-4d1f-9794-ba25679d0669 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:23.976208 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6b5036-9671-4d1f-9794-ba25679d0669 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:23.988091 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 914/1842] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:46:23 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:46:23.992044 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3d4756-cb34-4ea8-83ae-48df226dab10 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:24.005369 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3d4756-cb34-4ea8-83ae-48df226dab10 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:24.010158 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40c8237-b0a1-472b-8046-fc667684ba88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:24.010771 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40c8237-b0a1-472b-8046-fc667684ba88 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:24.011056 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40c8237-b0a1-472b-8046-fc667684ba88 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:24.011329 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40c8237-b0a1-472b-8046-fc667684ba88 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:24.011973 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40c8237-b0a1-472b-8046-fc667684ba88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:24.015258 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40c8237-b0a1-472b-8046-fc667684ba88 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:24.030292 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 929/1843] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:46:23 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:46:24.031071 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6b5036-9671-4d1f-9794-ba25679d0669 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:24.042857 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6b5036-9671-4d1f-9794-ba25679d0669 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:24.047618 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123764c2-498a-4c3e-bd83-f39342401aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:24.048443 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123764c2-498a-4c3e-bd83-f39342401aaf admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:24.049010 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123764c2-498a-4c3e-bd83-f39342401aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:24.049756 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123764c2-498a-4c3e-bd83-f39342401aaf admin admin] PATH_INFO: `/v3/projects/8b451a4727b341059e7ec44aafa4dcf9/users/5d9ab315cf834b129168df3404d0cf67/roles/3248f0268738489b883041851d681bb7` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:24.076040 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123764c2-498a-4c3e-bd83-f39342401aaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b451a4727b341059e7ec44aafa4dcf9, user_id=5d9ab315cf834b129168df3404d0cf67, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:24.078835 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123764c2-498a-4c3e-bd83-f39342401aaf admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:24.116341 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 915/1844] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:46:24 2022] PUT /identity/v3/projects/8b451a4727b341059e7ec44aafa4dcf9/users/5d9ab315cf834b129168df3404d0cf67/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:46:24.127735 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8dc39c-0c9c-4fa1-bcce-fb369ef5db07 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:24.127735 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8dc39c-0c9c-4fa1-bcce-fb369ef5db07 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:24.128045 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8dc39c-0c9c-4fa1-bcce-fb369ef5db07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:24.175479 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-0a8dc39c-0c9c-4fa1-bcce-fb369ef5db07 None None] MFA Rules not processed for user `5d9ab315cf834b129168df3404d0cf67`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:46:24.176970 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-0a8dc39c-0c9c-4fa1-bcce-fb369ef5db07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:46:24.265877 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 930/1845] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:46:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:46:24.270668 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123764c2-498a-4c3e-bd83-f39342401aaf admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:24.384993 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123764c2-498a-4c3e-bd83-f39342401aaf admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:24.392118 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aba11fb-7feb-470b-b789-c45897dc2041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:24.392845 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aba11fb-7feb-470b-b789-c45897dc2041 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:24.393150 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aba11fb-7feb-470b-b789-c45897dc2041 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:24.393540 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aba11fb-7feb-470b-b789-c45897dc2041 admin admin] PATH_INFO: `/v3/projects` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:24.394246 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aba11fb-7feb-470b-b789-c45897dc2041 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:24.397976 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aba11fb-7feb-470b-b789-c45897dc2041 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:24.444637 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 916/1846] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 08:46:24 2022] POST /identity/v3/projects => generated 387 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:46:24.446267 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8dc39c-0c9c-4fa1-bcce-fb369ef5db07 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:24.468557 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8dc39c-0c9c-4fa1-bcce-fb369ef5db07 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:24.505067 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:46:24.544888 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dce07bd-70aa-4698-8f4f-d019bfa3374e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:24.545635 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dce07bd-70aa-4698-8f4f-d019bfa3374e admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:24.546026 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dce07bd-70aa-4698-8f4f-d019bfa3374e admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:24.546518 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dce07bd-70aa-4698-8f4f-d019bfa3374e admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:24.547238 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dce07bd-70aa-4698-8f4f-d019bfa3374e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:24.549377 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dce07bd-70aa-4698-8f4f-d019bfa3374e admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:24.575012 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 931/1847] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:46:24 2022] POST /identity/v3/users => generated 300 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:46:24.578720 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aba11fb-7feb-470b-b789-c45897dc2041 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:24.591472 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aba11fb-7feb-470b-b789-c45897dc2041 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:24.596245 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d44c4f0-e555-4fc4-b15a-1b2c119c6f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:24.597077 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d44c4f0-e555-4fc4-b15a-1b2c119c6f11 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:24.597368 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d44c4f0-e555-4fc4-b15a-1b2c119c6f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:24.597645 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d44c4f0-e555-4fc4-b15a-1b2c119c6f11 admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:24.599884 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d44c4f0-e555-4fc4-b15a-1b2c119c6f11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:24.600017 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d44c4f0-e555-4fc4-b15a-1b2c119c6f11 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:24.612223 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 917/1848] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:46:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:46:24.616424 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dce07bd-70aa-4698-8f4f-d019bfa3374e admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:24.629123 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dce07bd-70aa-4698-8f4f-d019bfa3374e admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:24.637328 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ffb5fc-e64f-4b18-bf30-407a4066c69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:24.638056 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ffb5fc-e64f-4b18-bf30-407a4066c69f admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:24.638344 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ffb5fc-e64f-4b18-bf30-407a4066c69f admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:24.638623 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ffb5fc-e64f-4b18-bf30-407a4066c69f admin admin] PATH_INFO: `/v3/projects/87efdc5b15954b6cb53e30ad56491d11/users/045eed659b3c4e72b0e52acb2c553897/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:24.655552 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ffb5fc-e64f-4b18-bf30-407a4066c69f admin admin] RBAC: Authorizing `identity:create_grant(project_id=87efdc5b15954b6cb53e30ad56491d11, user_id=045eed659b3c4e72b0e52acb2c553897, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:24.658689 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ffb5fc-e64f-4b18-bf30-407a4066c69f admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:24.678749 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 932/1849] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:46:24 2022] PUT /identity/v3/projects/87efdc5b15954b6cb53e30ad56491d11/users/045eed659b3c4e72b0e52acb2c553897/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:46:24.682921 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d44c4f0-e555-4fc4-b15a-1b2c119c6f11 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:24.688544 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:46:24.696321 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d44c4f0-e555-4fc4-b15a-1b2c119c6f11 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:24.764697 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e41a55-30d2-44fd-a691-bfd4741bc606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:24.765316 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e41a55-30d2-44fd-a691-bfd4741bc606 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:24.765600 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e41a55-30d2-44fd-a691-bfd4741bc606 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:24.765922 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e41a55-30d2-44fd-a691-bfd4741bc606 admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:24.766570 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e41a55-30d2-44fd-a691-bfd4741bc606 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:24.768328 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e41a55-30d2-44fd-a691-bfd4741bc606 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:24.779380 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 918/1850] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:46:24 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:46:24.783698 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ffb5fc-e64f-4b18-bf30-407a4066c69f admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:24.794812 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ffb5fc-e64f-4b18-bf30-407a4066c69f admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:24.800157 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee0da20-82af-4e7b-9edb-be800c31d5c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:24.800851 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee0da20-82af-4e7b-9edb-be800c31d5c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:24.801131 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee0da20-82af-4e7b-9edb-be800c31d5c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:24.801440 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee0da20-82af-4e7b-9edb-be800c31d5c4 admin admin] PATH_INFO: `/v3/projects/87efdc5b15954b6cb53e30ad56491d11/users/045eed659b3c4e72b0e52acb2c553897/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:24.804016 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee0da20-82af-4e7b-9edb-be800c31d5c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87efdc5b15954b6cb53e30ad56491d11, user_id=045eed659b3c4e72b0e52acb2c553897, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:24.806761 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee0da20-82af-4e7b-9edb-be800c31d5c4 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:24.826463 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 933/1851] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:46:24 2022] PUT /identity/v3/projects/87efdc5b15954b6cb53e30ad56491d11/users/045eed659b3c4e72b0e52acb2c553897/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:46:24.837977 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c61d9f-7ca1-461b-976e-c7561d2e05cb None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:24.838268 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c61d9f-7ca1-461b-976e-c7561d2e05cb None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:24.839055 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c61d9f-7ca1-461b-976e-c7561d2e05cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:24.877429 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-68c61d9f-7ca1-461b-976e-c7561d2e05cb None None] MFA Rules not processed for user `045eed659b3c4e72b0e52acb2c553897`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:46:24.878820 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-68c61d9f-7ca1-461b-976e-c7561d2e05cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:46:24.969695 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 919/1852] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:46:24 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:46:24.981628 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee0da20-82af-4e7b-9edb-be800c31d5c4 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:24.983380 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-2ee0da20-82af-4e7b-9edb-be800c31d5c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:46:25.017236 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e2b79d-2d2f-4a79-b773-b07ebd88f30f None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:25.017879 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e2b79d-2d2f-4a79-b773-b07ebd88f30f None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:25.018854 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e2b79d-2d2f-4a79-b773-b07ebd88f30f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:25.041104 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-85e2b79d-2d2f-4a79-b773-b07ebd88f30f None None] MFA Rules not processed for user `f05cbab4640048589b9509fd9e8f655a`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:46:25.042350 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-85e2b79d-2d2f-4a79-b773-b07ebd88f30f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:46:25.051659 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee0da20-82af-4e7b-9edb-be800c31d5c4 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:25.057279 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab2a851-2b0a-4f89-9e08-fd35cce80eaa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:25.057944 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab2a851-2b0a-4f89-9e08-fd35cce80eaa service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:25.058228 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab2a851-2b0a-4f89-9e08-fd35cce80eaa service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:25.058504 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab2a851-2b0a-4f89-9e08-fd35cce80eaa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:25.067767 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab2a851-2b0a-4f89-9e08-fd35cce80eaa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:25.069234 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab2a851-2b0a-4f89-9e08-fd35cce80eaa service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:25.079526 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 934/1853] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:46:24 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:46:25.105217 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 920/1854] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:46:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 08:46:25.629645 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab2a851-2b0a-4f89-9e08-fd35cce80eaa service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:25.631855 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-9ab2a851-2b0a-4f89-9e08-fd35cce80eaa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:46:25.648072 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab2a851-2b0a-4f89-9e08-fd35cce80eaa service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:25.653397 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbb3989-9cbe-4588-b696-539602beaa74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:25.654036 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbb3989-9cbe-4588-b696-539602beaa74 service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:25.654432 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbb3989-9cbe-4588-b696-539602beaa74 service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:25.654588 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbb3989-9cbe-4588-b696-539602beaa74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:25.724593 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbb3989-9cbe-4588-b696-539602beaa74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:25.726806 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbb3989-9cbe-4588-b696-539602beaa74 service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:25.737890 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 935/1855] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:46:25 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:46:28.877698 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a35616-0f62-4ae2-9686-14e5ff3cb9b0 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:28.878174 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a35616-0f62-4ae2-9686-14e5ff3cb9b0 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:28.878265 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a35616-0f62-4ae2-9686-14e5ff3cb9b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:28.898095 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-20a35616-0f62-4ae2-9686-14e5ff3cb9b0 None None] MFA Rules not processed for user `c5b2a0e623c348ad94d4820080677be6`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:46:28.899930 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-20a35616-0f62-4ae2-9686-14e5ff3cb9b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:46:28.925692 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:46:28.957736 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 921/1856] 10.222.0.30 () {58 vars in 1009 bytes} [Sat Oct 15 08:46:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:46:28.970181 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbb3989-9cbe-4588-b696-539602beaa74 service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:28.971631 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-4cbb3989-9cbe-4588-b696-539602beaa74 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:46:29.003357 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:46:29.052495 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbb3989-9cbe-4588-b696-539602beaa74 service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:29.063305 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981ad4a6-ba59-460e-89a3-6b3fb6d8334e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e3009e20e4c49e5a74f7fd793b2c329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:29.064402 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981ad4a6-ba59-460e-89a3-6b3fb6d8334e service swift] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:29.064693 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981ad4a6-ba59-460e-89a3-6b3fb6d8334e service swift] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:29.065018 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981ad4a6-ba59-460e-89a3-6b3fb6d8334e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:29.074829 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981ad4a6-ba59-460e-89a3-6b3fb6d8334e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:29.076426 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981ad4a6-ba59-460e-89a3-6b3fb6d8334e service swift] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:29.085017 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 936/1857] 10.222.0.30 () {60 vars in 1384 bytes} [Sat Oct 15 08:46:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 08:46:29.962029 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a35616-0f62-4ae2-9686-14e5ff3cb9b0 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:29.976024 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a35616-0f62-4ae2-9686-14e5ff3cb9b0 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:29.979848 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8784ea0-3c6b-48af-ba24-e18f2a1d87f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:29.980481 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8784ea0-3c6b-48af-ba24-e18f2a1d87f8 service cinder] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:29.980768 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8784ea0-3c6b-48af-ba24-e18f2a1d87f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:29.981049 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8784ea0-3c6b-48af-ba24-e18f2a1d87f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:30.059776 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8784ea0-3c6b-48af-ba24-e18f2a1d87f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:30.061517 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8784ea0-3c6b-48af-ba24-e18f2a1d87f8 service cinder] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:30.072647 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 922/1858] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:46:29 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:46:32.367130 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981ad4a6-ba59-460e-89a3-6b3fb6d8334e service swift] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:32.460861 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981ad4a6-ba59-460e-89a3-6b3fb6d8334e service swift] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:32.466072 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f636452b-cef3-40b0-b05d-d96f9119edc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:32.466711 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f636452b-cef3-40b0-b05d-d96f9119edc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:32.466990 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f636452b-cef3-40b0-b05d-d96f9119edc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:32.467264 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f636452b-cef3-40b0-b05d-d96f9119edc1 admin admin] PATH_INFO: `/v3/users/1124027bf7264c0396c43e4ba435758a` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:32.468555 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f636452b-cef3-40b0-b05d-d96f9119edc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1124027bf7264c0396c43e4ba435758a)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:32.470519 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f636452b-cef3-40b0-b05d-d96f9119edc1 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:32.552749 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-f636452b-cef3-40b0-b05d-d96f9119edc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1124027bf7264c0396c43e4ba435758a'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:32.553494 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-f636452b-cef3-40b0-b05d-d96f9119edc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1124027bf7264c0396c43e4ba435758a'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:32.563460 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-f636452b-cef3-40b0-b05d-d96f9119edc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1124027bf7264c0396c43e4ba435758a'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:32.576624 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-f636452b-cef3-40b0-b05d-d96f9119edc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1124027bf7264c0396c43e4ba435758a'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:32.592196 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-f636452b-cef3-40b0-b05d-d96f9119edc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1124027bf7264c0396c43e4ba435758a'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:32.608356 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 937/1859] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 08:46:32 2022] DELETE /identity/v3/users/1124027bf7264c0396c43e4ba435758a => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:46:32.612847 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c37b25a-c3f5-418e-9a9c-962b93e5c688 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:32.613364 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c37b25a-c3f5-418e-9a9c-962b93e5c688 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:32.613471 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c37b25a-c3f5-418e-9a9c-962b93e5c688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:32.647835 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-1c37b25a-c3f5-418e-9a9c-962b93e5c688 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:46:32.649750 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-1c37b25a-c3f5-418e-9a9c-962b93e5c688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:46:32.729589 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 923/1860] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:46:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:46:32.741229 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f636452b-cef3-40b0-b05d-d96f9119edc1 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:32.742976 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-f636452b-cef3-40b0-b05d-d96f9119edc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:46:32.791082 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:46:32.815608 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f636452b-cef3-40b0-b05d-d96f9119edc1 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:32.819178 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9473c85f-d83a-457b-8c90-6e70d125bd49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:32.819867 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9473c85f-d83a-457b-8c90-6e70d125bd49 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:32.820217 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9473c85f-d83a-457b-8c90-6e70d125bd49 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:32.821007 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9473c85f-d83a-457b-8c90-6e70d125bd49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:32.830556 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9473c85f-d83a-457b-8c90-6e70d125bd49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:32.832066 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9473c85f-d83a-457b-8c90-6e70d125bd49 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:32.843198 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 938/1861] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:46:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:46:33.139178 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c37b25a-c3f5-418e-9a9c-962b93e5c688 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:33.140905 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-1c37b25a-c3f5-418e-9a9c-962b93e5c688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:46:33.158753 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c37b25a-c3f5-418e-9a9c-962b93e5c688 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:33.163918 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cdd4c2-df88-4028-8c8e-fb69a8da337f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:33.165211 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cdd4c2-df88-4028-8c8e-fb69a8da337f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:33.165556 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cdd4c2-df88-4028-8c8e-fb69a8da337f admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:33.165933 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cdd4c2-df88-4028-8c8e-fb69a8da337f admin admin] PATH_INFO: `/v3/projects/c14e5bb2fe8b46ca8cad7b981b042916` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:33.167167 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cdd4c2-df88-4028-8c8e-fb69a8da337f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c14e5bb2fe8b46ca8cad7b981b042916)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:33.170572 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cdd4c2-df88-4028-8c8e-fb69a8da337f admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:33.223893 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-12cdd4c2-df88-4028-8c8e-fb69a8da337f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c14e5bb2fe8b46ca8cad7b981b042916', 'request_id': 'req-12cdd4c2-df88-4028-8c8e-fb69a8da337f'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:33.230992 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-12cdd4c2-df88-4028-8c8e-fb69a8da337f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c14e5bb2fe8b46ca8cad7b981b042916', 'request_id': 'req-12cdd4c2-df88-4028-8c8e-fb69a8da337f'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:33.238071 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-12cdd4c2-df88-4028-8c8e-fb69a8da337f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c14e5bb2fe8b46ca8cad7b981b042916', 'request_id': 'req-12cdd4c2-df88-4028-8c8e-fb69a8da337f'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:33.242859 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:46:33.250290 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-12cdd4c2-df88-4028-8c8e-fb69a8da337f admin admin] The token cache is being invalidate because project c14e5bb2fe8b46ca8cad7b981b042916 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 08:46:33.250976 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-12cdd4c2-df88-4028-8c8e-fb69a8da337f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:33.252143 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-12cdd4c2-df88-4028-8c8e-fb69a8da337f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:33.253920 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 924/1862] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 08:46:33 2022] DELETE /identity/v3/projects/c14e5bb2fe8b46ca8cad7b981b042916 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:46:33.258811 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9473c85f-d83a-457b-8c90-6e70d125bd49 service neutron] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:33.269157 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9473c85f-d83a-457b-8c90-6e70d125bd49 service neutron] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:33.328302 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbb4ff1-75e8-48f1-b5fb-b33333178eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:33.328815 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb4ff1-75e8-48f1-b5fb-b33333178eb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:33.329151 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb4ff1-75e8-48f1-b5fb-b33333178eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:33.329383 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb4ff1-75e8-48f1-b5fb-b33333178eb3 admin admin] PATH_INFO: `/v3/users/5d9ab315cf834b129168df3404d0cf67` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:33.330765 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbb4ff1-75e8-48f1-b5fb-b33333178eb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d9ab315cf834b129168df3404d0cf67)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:33.332809 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbb4ff1-75e8-48f1-b5fb-b33333178eb3 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:33.413543 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-6cbb4ff1-75e8-48f1-b5fb-b33333178eb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d9ab315cf834b129168df3404d0cf67'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:33.413543 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-6cbb4ff1-75e8-48f1-b5fb-b33333178eb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d9ab315cf834b129168df3404d0cf67'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:33.437201 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-6cbb4ff1-75e8-48f1-b5fb-b33333178eb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d9ab315cf834b129168df3404d0cf67'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:33.456030 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-6cbb4ff1-75e8-48f1-b5fb-b33333178eb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d9ab315cf834b129168df3404d0cf67'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:33.465388 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-6cbb4ff1-75e8-48f1-b5fb-b33333178eb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d9ab315cf834b129168df3404d0cf67'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:33.472615 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 939/1863] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 08:46:33 2022] DELETE /identity/v3/users/5d9ab315cf834b129168df3404d0cf67 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:46:33.846185 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cdd4c2-df88-4028-8c8e-fb69a8da337f admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:33.848422 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-12cdd4c2-df88-4028-8c8e-fb69a8da337f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:46:33.897496 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cdd4c2-df88-4028-8c8e-fb69a8da337f admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:33.969061 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f86a23-5b22-48f2-976e-ec20b004383e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:33.969942 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f86a23-5b22-48f2-976e-ec20b004383e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:33.970202 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f86a23-5b22-48f2-976e-ec20b004383e admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:33.970756 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f86a23-5b22-48f2-976e-ec20b004383e admin admin] PATH_INFO: `/v3/projects/8b451a4727b341059e7ec44aafa4dcf9` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:33.971895 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f86a23-5b22-48f2-976e-ec20b004383e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b451a4727b341059e7ec44aafa4dcf9)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:33.973878 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f86a23-5b22-48f2-976e-ec20b004383e admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:34.037454 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-60f86a23-5b22-48f2-976e-ec20b004383e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b451a4727b341059e7ec44aafa4dcf9', 'request_id': 'req-60f86a23-5b22-48f2-976e-ec20b004383e'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:34.044289 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-60f86a23-5b22-48f2-976e-ec20b004383e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b451a4727b341059e7ec44aafa4dcf9', 'request_id': 'req-60f86a23-5b22-48f2-976e-ec20b004383e'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:34.051582 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-60f86a23-5b22-48f2-976e-ec20b004383e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b451a4727b341059e7ec44aafa4dcf9', 'request_id': 'req-60f86a23-5b22-48f2-976e-ec20b004383e'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:34.062530 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-60f86a23-5b22-48f2-976e-ec20b004383e admin admin] The token cache is being invalidate because project 8b451a4727b341059e7ec44aafa4dcf9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 08:46:34.062823 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-60f86a23-5b22-48f2-976e-ec20b004383e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:34.063449 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-60f86a23-5b22-48f2-976e-ec20b004383e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:34.064935 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 925/1864] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 08:46:33 2022] DELETE /identity/v3/projects/8b451a4727b341059e7ec44aafa4dcf9 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:46:34.070011 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbb4ff1-75e8-48f1-b5fb-b33333178eb3 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:34.072075 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-6cbb4ff1-75e8-48f1-b5fb-b33333178eb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:46:34.085327 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbb4ff1-75e8-48f1-b5fb-b33333178eb3 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:34.146629 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:46:34.154760 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3e4ebb-3566-4fb4-8513-0dec5464ac46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:34.155534 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e4ebb-3566-4fb4-8513-0dec5464ac46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:34.155861 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e4ebb-3566-4fb4-8513-0dec5464ac46 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:34.156141 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e4ebb-3566-4fb4-8513-0dec5464ac46 admin admin] PATH_INFO: `/v3/users/045eed659b3c4e72b0e52acb2c553897` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:34.157722 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3e4ebb-3566-4fb4-8513-0dec5464ac46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=045eed659b3c4e72b0e52acb2c553897)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:34.159693 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3e4ebb-3566-4fb4-8513-0dec5464ac46 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:34.213311 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-de3e4ebb-3566-4fb4-8513-0dec5464ac46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '045eed659b3c4e72b0e52acb2c553897'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:34.214125 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-de3e4ebb-3566-4fb4-8513-0dec5464ac46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '045eed659b3c4e72b0e52acb2c553897'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:34.221620 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-de3e4ebb-3566-4fb4-8513-0dec5464ac46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '045eed659b3c4e72b0e52acb2c553897'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:34.234121 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-de3e4ebb-3566-4fb4-8513-0dec5464ac46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '045eed659b3c4e72b0e52acb2c553897'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:34.242835 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-de3e4ebb-3566-4fb4-8513-0dec5464ac46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '045eed659b3c4e72b0e52acb2c553897'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:34.249249 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 940/1865] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 08:46:34 2022] DELETE /identity/v3/users/045eed659b3c4e72b0e52acb2c553897 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 08:46:34.544627 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f86a23-5b22-48f2-976e-ec20b004383e admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:34.546092 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-60f86a23-5b22-48f2-976e-ec20b004383e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:46:34.562974 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f86a23-5b22-48f2-976e-ec20b004383e admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:34.636382 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c6f143-6985-471c-8b7b-a2a029f50e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:34.637074 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6f143-6985-471c-8b7b-a2a029f50e8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:34.637360 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6f143-6985-471c-8b7b-a2a029f50e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:34.637640 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6f143-6985-471c-8b7b-a2a029f50e8c admin admin] PATH_INFO: `/v3/projects/87efdc5b15954b6cb53e30ad56491d11` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:34.638832 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c6f143-6985-471c-8b7b-a2a029f50e8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=87efdc5b15954b6cb53e30ad56491d11)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:34.640743 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c6f143-6985-471c-8b7b-a2a029f50e8c admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:34.693916 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:46:34.713756 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-33c6f143-6985-471c-8b7b-a2a029f50e8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87efdc5b15954b6cb53e30ad56491d11', 'request_id': 'req-33c6f143-6985-471c-8b7b-a2a029f50e8c'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:34.720703 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-33c6f143-6985-471c-8b7b-a2a029f50e8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87efdc5b15954b6cb53e30ad56491d11', 'request_id': 'req-33c6f143-6985-471c-8b7b-a2a029f50e8c'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:34.728450 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-33c6f143-6985-471c-8b7b-a2a029f50e8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87efdc5b15954b6cb53e30ad56491d11', 'request_id': 'req-33c6f143-6985-471c-8b7b-a2a029f50e8c'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:34.739797 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-33c6f143-6985-471c-8b7b-a2a029f50e8c admin admin] The token cache is being invalidate because project 87efdc5b15954b6cb53e30ad56491d11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 08:46:34.740105 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-33c6f143-6985-471c-8b7b-a2a029f50e8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:34.740751 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-33c6f143-6985-471c-8b7b-a2a029f50e8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:46:34.747336 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 926/1866] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 08:46:34 2022] DELETE /identity/v3/projects/87efdc5b15954b6cb53e30ad56491d11 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:46:34.749020 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df57f3d-2a22-4115-922a-61fe3ac2a57b None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:34.749496 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df57f3d-2a22-4115-922a-61fe3ac2a57b None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:34.749597 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df57f3d-2a22-4115-922a-61fe3ac2a57b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:34.772654 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-5df57f3d-2a22-4115-922a-61fe3ac2a57b None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:46:34.774048 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-5df57f3d-2a22-4115-922a-61fe3ac2a57b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:46:34.845931 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 941/1867] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:46:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:46:34.855015 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2411a1e-6e19-4179-91e1-452b67be3305 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:34.855593 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2411a1e-6e19-4179-91e1-452b67be3305 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:34.855593 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2411a1e-6e19-4179-91e1-452b67be3305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:34.878248 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-b2411a1e-6e19-4179-91e1-452b67be3305 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:46:34.879585 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-b2411a1e-6e19-4179-91e1-452b67be3305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:46:34.889729 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 927/1868] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:46:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:46:34.892433 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df57f3d-2a22-4115-922a-61fe3ac2a57b None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:34.907386 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df57f3d-2a22-4115-922a-61fe3ac2a57b None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:34.912432 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff01706-1463-467f-863e-bcf9d61ef0c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:34.913304 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff01706-1463-467f-863e-bcf9d61ef0c7 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:34.913702 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff01706-1463-467f-863e-bcf9d61ef0c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:34.914651 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff01706-1463-467f-863e-bcf9d61ef0c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:34.915416 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff01706-1463-467f-863e-bcf9d61ef0c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:46:34.915912 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff01706-1463-467f-863e-bcf9d61ef0c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:34.923019 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff01706-1463-467f-863e-bcf9d61ef0c7 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:34.942709 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 942/1869] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 08:46:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 08:46:34.947188 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2411a1e-6e19-4179-91e1-452b67be3305 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:34.957051 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2411a1e-6e19-4179-91e1-452b67be3305 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:34.962642 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f395f136-af8b-4ad3-8a53-2092b64f77d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:34.963255 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395f136-af8b-4ad3-8a53-2092b64f77d3 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:34.963637 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395f136-af8b-4ad3-8a53-2092b64f77d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:34.963916 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395f136-af8b-4ad3-8a53-2092b64f77d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:34.964657 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f395f136-af8b-4ad3-8a53-2092b64f77d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:34.967282 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f395f136-af8b-4ad3-8a53-2092b64f77d3 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:35.001724 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 928/1870] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 08:46:34 2022] POST /identity/v3/projects => generated 377 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:46:35.004574 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff01706-1463-467f-863e-bcf9d61ef0c7 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:35.015483 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff01706-1463-467f-863e-bcf9d61ef0c7 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:35.081878 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cfaba8-64fe-4196-af72-a56835f3d046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:35.082759 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cfaba8-64fe-4196-af72-a56835f3d046 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:35.083228 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cfaba8-64fe-4196-af72-a56835f3d046 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:35.083652 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cfaba8-64fe-4196-af72-a56835f3d046 admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:35.084612 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cfaba8-64fe-4196-af72-a56835f3d046 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:35.087462 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cfaba8-64fe-4196-af72-a56835f3d046 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:35.098828 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:46:35.121819 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 943/1871] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:46:35 2022] POST /identity/v3/users => generated 289 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 08:46:35.123043 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f395f136-af8b-4ad3-8a53-2092b64f77d3 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:35.135847 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f395f136-af8b-4ad3-8a53-2092b64f77d3 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:35.142634 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85880cd-b01a-4a98-b7ee-f632b344c7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:35.143284 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85880cd-b01a-4a98-b7ee-f632b344c7ab admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:35.143567 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85880cd-b01a-4a98-b7ee-f632b344c7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:35.143934 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85880cd-b01a-4a98-b7ee-f632b344c7ab admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:35.144539 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85880cd-b01a-4a98-b7ee-f632b344c7ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:35.146263 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85880cd-b01a-4a98-b7ee-f632b344c7ab admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:35.158110 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 929/1872] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:46:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 08:46:35.162388 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cfaba8-64fe-4196-af72-a56835f3d046 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:35.172917 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cfaba8-64fe-4196-af72-a56835f3d046 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:35.178116 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60076509-29df-485f-85b7-4fb654b98d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:35.178825 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60076509-29df-485f-85b7-4fb654b98d2a admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:35.179189 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60076509-29df-485f-85b7-4fb654b98d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:35.179542 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60076509-29df-485f-85b7-4fb654b98d2a admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:35.180740 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60076509-29df-485f-85b7-4fb654b98d2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:35.182435 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60076509-29df-485f-85b7-4fb654b98d2a admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:35.196732 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 944/1873] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:46:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 08:46:35.197938 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85880cd-b01a-4a98-b7ee-f632b344c7ab admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:35.207606 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85880cd-b01a-4a98-b7ee-f632b344c7ab admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:35.212334 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5e4ad6-1b73-4570-a676-a8c4d2dcce12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:35.213017 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e4ad6-1b73-4570-a676-a8c4d2dcce12 admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:35.213298 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e4ad6-1b73-4570-a676-a8c4d2dcce12 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:35.213570 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e4ad6-1b73-4570-a676-a8c4d2dcce12 admin admin] PATH_INFO: `/v3/projects/4127b82b10e24d01aaa211bb70f236e9/users/f6375453adf049d29facbbf8733ab1c4/roles/3248f0268738489b883041851d681bb7` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:35.240577 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5e4ad6-1b73-4570-a676-a8c4d2dcce12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4127b82b10e24d01aaa211bb70f236e9, user_id=f6375453adf049d29facbbf8733ab1c4, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:35.244218 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5e4ad6-1b73-4570-a676-a8c4d2dcce12 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:35.259372 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 930/1874] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:46:35 2022] PUT /identity/v3/projects/4127b82b10e24d01aaa211bb70f236e9/users/f6375453adf049d29facbbf8733ab1c4/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:46:35.270950 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ecebf-5890-4cc6-b082-2abfc378c957 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:35.271175 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ecebf-5890-4cc6-b082-2abfc378c957 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:35.271462 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ecebf-5890-4cc6-b082-2abfc378c957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:35.301837 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-229ecebf-5890-4cc6-b082-2abfc378c957 None None] MFA Rules not processed for user `f6375453adf049d29facbbf8733ab1c4`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:46:35.303072 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-229ecebf-5890-4cc6-b082-2abfc378c957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:46:35.377737 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 945/1875] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:46:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:46:35.385358 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5e4ad6-1b73-4570-a676-a8c4d2dcce12 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:46:35.387647 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-ff5e4ad6-1b73-4570-a676-a8c4d2dcce12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:46:35.468808 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5e4ad6-1b73-4570-a676-a8c4d2dcce12 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:46:35.473599 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8a1342-8261-4687-bc2a-ffcefd779bb7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71692d05fd324679b1598feb52b3fe6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:46:35.474246 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8a1342-8261-4687-bc2a-ffcefd779bb7 service glance] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:46:35.474525 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8a1342-8261-4687-bc2a-ffcefd779bb7 service glance] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:46:35.474795 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8a1342-8261-4687-bc2a-ffcefd779bb7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:46:35.484199 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8a1342-8261-4687-bc2a-ffcefd779bb7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:46:35.485650 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8a1342-8261-4687-bc2a-ffcefd779bb7 service glance] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:46:35.494492 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 931/1876] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:46:35 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 08:46:45.374091 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:INSERT=1,keystone:SELECT=39 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:46:45.490311 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=67,keystone:DELETE=5,keystone:INSERT=3 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:47:46.154374 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229ecebf-5890-4cc6-b082-2abfc378c957 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:47:46.155794 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-229ecebf-5890-4cc6-b082-2abfc378c957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:47:46.246016 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229ecebf-5890-4cc6-b082-2abfc378c957 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:47:46.251131 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a8fe5b-c2e1-46e0-998c-9828b38c89f3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38986ca2612c4fc59d960cd693834f9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:47:46.251846 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a8fe5b-c2e1-46e0-998c-9828b38c89f3 service placement] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:47:46.252136 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a8fe5b-c2e1-46e0-998c-9828b38c89f3 service placement] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:47:46.252414 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a8fe5b-c2e1-46e0-998c-9828b38c89f3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:47:46.254192 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-94a8fe5b-c2e1-46e0-998c-9828b38c89f3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:47:46.266998 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a8fe5b-c2e1-46e0-998c-9828b38c89f3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:47:46.268721 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a8fe5b-c2e1-46e0-998c-9828b38c89f3 service placement] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:47:46.280979 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 946/1877] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:47:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 08:47:46.872547 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad819720-b60c-4c17-9871-a79c868a9729 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:47:46.873815 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad819720-b60c-4c17-9871-a79c868a9729 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:47:46.874581 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad819720-b60c-4c17-9871-a79c868a9729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:47:46.903894 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-ad819720-b60c-4c17-9871-a79c868a9729 None None] MFA Rules not processed for user `c5b2a0e623c348ad94d4820080677be6`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:47:46.905084 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-ad819720-b60c-4c17-9871-a79c868a9729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:47:46.971920 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 932/1878] 10.222.0.30 () {58 vars in 1009 bytes} [Sat Oct 15 08:47:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:47:47.013379 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a8fe5b-c2e1-46e0-998c-9828b38c89f3 service placement] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:47:47.014812 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-94a8fe5b-c2e1-46e0-998c-9828b38c89f3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:47:47.084648 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a8fe5b-c2e1-46e0-998c-9828b38c89f3 service placement] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:47:47.094491 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787b3248-d8a2-410f-b3c7-edf90f60afa5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e3009e20e4c49e5a74f7fd793b2c329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:47:47.095208 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b3248-d8a2-410f-b3c7-edf90f60afa5 service swift] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:47:47.095501 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b3248-d8a2-410f-b3c7-edf90f60afa5 service swift] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:47:47.095782 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b3248-d8a2-410f-b3c7-edf90f60afa5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:47:47.105149 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787b3248-d8a2-410f-b3c7-edf90f60afa5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:47:47.106805 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787b3248-d8a2-410f-b3c7-edf90f60afa5 service swift] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:47:47.118966 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 947/1879] 10.222.0.30 () {60 vars in 1384 bytes} [Sat Oct 15 08:47:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 08:47:56.961090 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=25 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:47:57.112023 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=64 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:48:02.843521 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda1258e-777c-4d99-b1d8-6eb80ffed8b4 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:48:02.844057 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda1258e-777c-4d99-b1d8-6eb80ffed8b4 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:48:02.845664 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda1258e-777c-4d99-b1d8-6eb80ffed8b4 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:48:02.848140 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 933/1880] 10.222.0.30 () {54 vars in 921 bytes} [Sat Oct 15 08:48:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:48:02.856660 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4944df-58cb-4f06-ad6e-c425d65a151b None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:48:02.857593 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4944df-58cb-4f06-ad6e-c425d65a151b None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:48:02.858108 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4944df-58cb-4f06-ad6e-c425d65a151b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:48:02.880456 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-bf4944df-58cb-4f06-ad6e-c425d65a151b None None] MFA Rules not processed for user `af0a8af5cadc4cc1ab73e8eb2a20bb7e`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:48:02.881689 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-bf4944df-58cb-4f06-ad6e-c425d65a151b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:48:02.959106 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 948/1881] 10.222.0.30 () {58 vars in 1018 bytes} [Sat Oct 15 08:48:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 08:48:02.970677 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda1258e-777c-4d99-b1d8-6eb80ffed8b4 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:48:02.972447 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-bda1258e-777c-4d99-b1d8-6eb80ffed8b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:48:02.986366 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda1258e-777c-4d99-b1d8-6eb80ffed8b4 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:48:02.997284 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b524e2-05a1-4768-a4d9-332e8b5b9114 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71692d05fd324679b1598feb52b3fe6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:48:02.998198 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b524e2-05a1-4768-a4d9-332e8b5b9114 service glance] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:48:02.998577 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b524e2-05a1-4768-a4d9-332e8b5b9114 service glance] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:48:02.998860 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b524e2-05a1-4768-a4d9-332e8b5b9114 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:48:03.009514 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b524e2-05a1-4768-a4d9-332e8b5b9114 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:48:03.011445 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b524e2-05a1-4768-a4d9-332e8b5b9114 service glance] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:48:03.022428 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 934/1882] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:48:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:48:03.091151 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590eb705-04d2-475c-a50e-e2f8c023f358 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:48:03.091443 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590eb705-04d2-475c-a50e-e2f8c023f358 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:48:03.091790 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590eb705-04d2-475c-a50e-e2f8c023f358 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:48:03.093155 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 949/1883] 10.222.0.30 () {54 vars in 921 bytes} [Sat Oct 15 08:48:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:48:03.097973 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe452e-c4d7-43d4-8451-29c64b3c39cf None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:48:03.098264 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe452e-c4d7-43d4-8451-29c64b3c39cf None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:48:03.098541 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe452e-c4d7-43d4-8451-29c64b3c39cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:48:03.117653 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-94fe452e-c4d7-43d4-8451-29c64b3c39cf None None] MFA Rules not processed for user `8b11e349e4cb4558a4c75d843effc0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:48:03.119429 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-94fe452e-c4d7-43d4-8451-29c64b3c39cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:48:03.170961 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 935/1884] 10.222.0.30 () {58 vars in 1018 bytes} [Sat Oct 15 08:48:03 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:48:03.183782 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590eb705-04d2-475c-a50e-e2f8c023f358 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:48:03.185479 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-590eb705-04d2-475c-a50e-e2f8c023f358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:48:03.198446 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590eb705-04d2-475c-a50e-e2f8c023f358 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:48:03.201935 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bc9e79-6ab9-418a-b057-3b678920aeef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:48:03.202533 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc9e79-6ab9-418a-b057-3b678920aeef service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:48:03.202817 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc9e79-6ab9-418a-b057-3b678920aeef service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:48:03.203093 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc9e79-6ab9-418a-b057-3b678920aeef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:48:03.209948 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bc9e79-6ab9-418a-b057-3b678920aeef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:48:03.211359 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bc9e79-6ab9-418a-b057-3b678920aeef service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:48:03.218995 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 950/1885] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:48:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:48:03.330559 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55d4cc-d22a-42a8-b79f-a55ad57e79d2 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:48:03.330842 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55d4cc-d22a-42a8-b79f-a55ad57e79d2 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:48:03.331114 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55d4cc-d22a-42a8-b79f-a55ad57e79d2 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:48:03.332468 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 936/1886] 10.222.0.30 () {54 vars in 921 bytes} [Sat Oct 15 08:48:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:48:03.337845 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c52290-6759-4fed-96d1-49b5b4c66775 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:48:03.338133 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c52290-6759-4fed-96d1-49b5b4c66775 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:48:03.338411 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c52290-6759-4fed-96d1-49b5b4c66775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:48:03.358203 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-52c52290-6759-4fed-96d1-49b5b4c66775 None None] MFA Rules not processed for user `af0a8af5cadc4cc1ab73e8eb2a20bb7e`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:48:03.359429 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-52c52290-6759-4fed-96d1-49b5b4c66775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:48:03.368552 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 951/1887] 10.222.0.30 () {58 vars in 1018 bytes} [Sat Oct 15 08:48:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 08:48:03.379368 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd55d4cc-d22a-42a8-b79f-a55ad57e79d2 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:48:03.390146 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd55d4cc-d22a-42a8-b79f-a55ad57e79d2 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:48:03.393585 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b6d5d7-f8ff-42b5-8e3c-35a49ddc425e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:48:03.395656 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6d5d7-f8ff-42b5-8e3c-35a49ddc425e service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:48:03.395656 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6d5d7-f8ff-42b5-8e3c-35a49ddc425e service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:48:03.395656 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6d5d7-f8ff-42b5-8e3c-35a49ddc425e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:48:03.405084 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b6d5d7-f8ff-42b5-8e3c-35a49ddc425e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:48:03.406541 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b6d5d7-f8ff-42b5-8e3c-35a49ddc425e service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:48:03.417453 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 937/1888] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:48:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:48:13.352752 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=40 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:48:13.412274 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=35 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:49:26.256039 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c52290-6759-4fed-96d1-49b5b4c66775 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:49:26.257558 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-52c52290-6759-4fed-96d1-49b5b4c66775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:49:26.269077 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c52290-6759-4fed-96d1-49b5b4c66775 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:49:26.272567 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3a2646-5602-4272-8f10-94c7d2343d25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:49:26.273169 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3a2646-5602-4272-8f10-94c7d2343d25 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:49:26.273448 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3a2646-5602-4272-8f10-94c7d2343d25 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:49:26.273754 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3a2646-5602-4272-8f10-94c7d2343d25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:49:26.275437 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-ef3a2646-5602-4272-8f10-94c7d2343d25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:49:26.288925 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3a2646-5602-4272-8f10-94c7d2343d25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:49:26.290368 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3a2646-5602-4272-8f10-94c7d2343d25 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:49:26.300968 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 952/1889] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:49:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:49:27.482988 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b6d5d7-f8ff-42b5-8e3c-35a49ddc425e service neutron] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:49:27.498289 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b6d5d7-f8ff-42b5-8e3c-35a49ddc425e service neutron] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:49:27.503196 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caac00ba-c389-4b52-b191-86ddab2b000b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:49:27.503804 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caac00ba-c389-4b52-b191-86ddab2b000b service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:49:27.504085 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caac00ba-c389-4b52-b191-86ddab2b000b service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:49:27.504361 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caac00ba-c389-4b52-b191-86ddab2b000b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:49:27.506183 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-caac00ba-c389-4b52-b191-86ddab2b000b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:49:27.520390 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caac00ba-c389-4b52-b191-86ddab2b000b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:49:27.521878 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caac00ba-c389-4b52-b191-86ddab2b000b service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:49:27.533989 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 938/1890] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:49:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:49:32.955897 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3a2646-5602-4272-8f10-94c7d2343d25 service neutron] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:49:32.957304 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-ef3a2646-5602-4272-8f10-94c7d2343d25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:49:33.014630 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3a2646-5602-4272-8f10-94c7d2343d25 service neutron] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:49:33.018315 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e618c239-c0ad-4ecd-96d9-70b49c2eeac4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:49:33.018921 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e618c239-c0ad-4ecd-96d9-70b49c2eeac4 service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:49:33.019204 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e618c239-c0ad-4ecd-96d9-70b49c2eeac4 service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:49:33.019480 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e618c239-c0ad-4ecd-96d9-70b49c2eeac4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:49:33.021165 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-e618c239-c0ad-4ecd-96d9-70b49c2eeac4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:49:33.079096 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e618c239-c0ad-4ecd-96d9-70b49c2eeac4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:49:33.080510 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e618c239-c0ad-4ecd-96d9-70b49c2eeac4 service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:49:33.089355 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 953/1891] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:49:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:49:37.527985 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=6 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:49:40.540989 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caac00ba-c389-4b52-b191-86ddab2b000b service neutron] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:49:40.550947 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caac00ba-c389-4b52-b191-86ddab2b000b service neutron] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:49:40.554526 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b7e05c-f2ff-466b-833e-b70d93163e06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:49:40.555125 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b7e05c-f2ff-466b-833e-b70d93163e06 service cinder] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:49:40.555429 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b7e05c-f2ff-466b-833e-b70d93163e06 service cinder] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:49:40.555728 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b7e05c-f2ff-466b-833e-b70d93163e06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:49:40.557486 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-12b7e05c-f2ff-466b-833e-b70d93163e06 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:49:40.616428 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b7e05c-f2ff-466b-833e-b70d93163e06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:49:40.617901 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b7e05c-f2ff-466b-833e-b70d93163e06 service cinder] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:49:40.627375 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 939/1892] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:49:40 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:49:43.085182 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=54 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:49:45.164934 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e618c239-c0ad-4ecd-96d9-70b49c2eeac4 service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:49:45.166299 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-e618c239-c0ad-4ecd-96d9-70b49c2eeac4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:49:45.181490 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e618c239-c0ad-4ecd-96d9-70b49c2eeac4 service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:49:45.186911 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd900415-69c2-4d89-bb4b-cf033948b8a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0a8af5cadc4cc1ab73e8eb2a20bb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:49:45.187504 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd900415-69c2-4d89-bb4b-cf033948b8a0 service nova] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:49:45.187788 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd900415-69c2-4d89-bb4b-cf033948b8a0 service nova] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:49:45.188065 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd900415-69c2-4d89-bb4b-cf033948b8a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:49:45.189741 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-bd900415-69c2-4d89-bb4b-cf033948b8a0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:49:45.247137 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd900415-69c2-4d89-bb4b-cf033948b8a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:49:45.248707 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd900415-69c2-4d89-bb4b-cf033948b8a0 service nova] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:49:45.257936 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 954/1893] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:49:45 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:49:46.480663 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b7e05c-f2ff-466b-833e-b70d93163e06 service cinder] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:49:46.482133 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-12b7e05c-f2ff-466b-833e-b70d93163e06 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:49:46.493920 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b7e05c-f2ff-466b-833e-b70d93163e06 service cinder] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:49:46.497521 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31c9954-1567-4748-ad16-2baafa7eae17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:49:46.498731 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31c9954-1567-4748-ad16-2baafa7eae17 service cinder] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:49:46.499010 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31c9954-1567-4748-ad16-2baafa7eae17 service cinder] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:49:46.499306 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31c9954-1567-4748-ad16-2baafa7eae17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:49:46.501086 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-f31c9954-1567-4748-ad16-2baafa7eae17 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:49:46.560439 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31c9954-1567-4748-ad16-2baafa7eae17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:49:46.562561 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31c9954-1567-4748-ad16-2baafa7eae17 service cinder] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:49:46.571872 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 940/1894] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:49:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:49:55.253548 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=28 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:49:56.567459 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=56 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:50:43.734889 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd900415-69c2-4d89-bb4b-cf033948b8a0 service nova] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:50:43.743960 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd900415-69c2-4d89-bb4b-cf033948b8a0 service nova] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:50:43.747527 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83416d45-fd6b-4286-9821-f86473525dd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:50:43.748127 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83416d45-fd6b-4286-9821-f86473525dd7 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:50:43.748412 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83416d45-fd6b-4286-9821-f86473525dd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:50:43.748689 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83416d45-fd6b-4286-9821-f86473525dd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:50:43.750416 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-83416d45-fd6b-4286-9821-f86473525dd7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:50:43.759640 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83416d45-fd6b-4286-9821-f86473525dd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:50:43.761050 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83416d45-fd6b-4286-9821-f86473525dd7 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:50:43.769224 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 955/1895] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:50:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:50:53.765852 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=6 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:51:01.422152 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31c9954-1567-4748-ad16-2baafa7eae17 service cinder] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:51:01.430798 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31c9954-1567-4748-ad16-2baafa7eae17 service cinder] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:51:01.434446 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b894da-ed79-4e0b-bf5a-d9eb6adbdd3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:51:01.435042 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b894da-ed79-4e0b-bf5a-d9eb6adbdd3b service cinder] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:51:01.435324 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b894da-ed79-4e0b-bf5a-d9eb6adbdd3b service cinder] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:51:01.435606 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b894da-ed79-4e0b-bf5a-d9eb6adbdd3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:51:01.437367 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-38b894da-ed79-4e0b-bf5a-d9eb6adbdd3b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:51:01.494956 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b894da-ed79-4e0b-bf5a-d9eb6adbdd3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:51:01.496394 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b894da-ed79-4e0b-bf5a-d9eb6adbdd3b service cinder] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:51:01.505107 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 941/1896] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:51:01 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:51:11.501045 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=28 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:51:14.156345 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83416d45-fd6b-4286-9821-f86473525dd7 service neutron] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:51:14.166100 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83416d45-fd6b-4286-9821-f86473525dd7 service neutron] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:51:14.172011 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7effc57c-c4e2-4088-a51a-a59d433062f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:51:14.172611 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7effc57c-c4e2-4088-a51a-a59d433062f3 service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:51:14.172894 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7effc57c-c4e2-4088-a51a-a59d433062f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:51:14.173171 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7effc57c-c4e2-4088-a51a-a59d433062f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:51:14.174989 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-7effc57c-c4e2-4088-a51a-a59d433062f3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:51:14.235130 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7effc57c-c4e2-4088-a51a-a59d433062f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:51:14.236478 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7effc57c-c4e2-4088-a51a-a59d433062f3 service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:51:14.245302 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 956/1897] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:51:14 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:51:24.241179 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=28 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:51:36.116681 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b894da-ed79-4e0b-bf5a-d9eb6adbdd3b service cinder] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:51:36.125958 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b894da-ed79-4e0b-bf5a-d9eb6adbdd3b service cinder] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:51:36.130331 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00301054-8908-4d63-b83e-01ca7efd2aee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:51:36.130953 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00301054-8908-4d63-b83e-01ca7efd2aee service cinder] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:51:36.131237 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00301054-8908-4d63-b83e-01ca7efd2aee service cinder] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:51:36.131514 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00301054-8908-4d63-b83e-01ca7efd2aee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:51:36.140611 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00301054-8908-4d63-b83e-01ca7efd2aee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:51:36.142155 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00301054-8908-4d63-b83e-01ca7efd2aee service cinder] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:51:36.151960 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 942/1898] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:51:36 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:51:46.147456 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=6 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:52:36.271492 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7effc57c-c4e2-4088-a51a-a59d433062f3 service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:52:36.380299 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7effc57c-c4e2-4088-a51a-a59d433062f3 service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:52:36.384851 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057c4371-bf5a-4835-95d6-b09c7cebb79d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:52:36.385487 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c4371-bf5a-4835-95d6-b09c7cebb79d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:52:36.385806 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c4371-bf5a-4835-95d6-b09c7cebb79d admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:52:36.386085 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c4371-bf5a-4835-95d6-b09c7cebb79d admin admin] PATH_INFO: `/v3/users/f6375453adf049d29facbbf8733ab1c4` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:52:36.387218 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057c4371-bf5a-4835-95d6-b09c7cebb79d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6375453adf049d29facbbf8733ab1c4)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:52:36.389164 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057c4371-bf5a-4835-95d6-b09c7cebb79d admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:52:36.434958 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-057c4371-bf5a-4835-95d6-b09c7cebb79d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6375453adf049d29facbbf8733ab1c4'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:52:36.435603 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-057c4371-bf5a-4835-95d6-b09c7cebb79d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6375453adf049d29facbbf8733ab1c4'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:52:36.442919 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-057c4371-bf5a-4835-95d6-b09c7cebb79d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6375453adf049d29facbbf8733ab1c4'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:52:36.455252 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-057c4371-bf5a-4835-95d6-b09c7cebb79d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6375453adf049d29facbbf8733ab1c4'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:52:36.462517 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-057c4371-bf5a-4835-95d6-b09c7cebb79d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6375453adf049d29facbbf8733ab1c4'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:52:36.468385 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 957/1899] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 08:52:36 2022] DELETE /identity/v3/users/f6375453adf049d29facbbf8733ab1c4 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:52:36.473494 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63a2f5-c28b-4b9a-b170-1d8873686b9c None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:52:36.474003 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63a2f5-c28b-4b9a-b170-1d8873686b9c None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:52:36.474109 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63a2f5-c28b-4b9a-b170-1d8873686b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:52:36.521019 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-ba63a2f5-c28b-4b9a-b170-1d8873686b9c None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:52:36.522241 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-ba63a2f5-c28b-4b9a-b170-1d8873686b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:52:36.580410 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 943/1900] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:52:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:52:36.590686 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057c4371-bf5a-4835-95d6-b09c7cebb79d admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:52:36.592078 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-057c4371-bf5a-4835-95d6-b09c7cebb79d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:52:36.633021 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:52:36.667255 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057c4371-bf5a-4835-95d6-b09c7cebb79d admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:52:36.671538 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ea2499-ac6e-45d8-9373-62e297f9fa10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:52:36.672182 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ea2499-ac6e-45d8-9373-62e297f9fa10 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:52:36.672471 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ea2499-ac6e-45d8-9373-62e297f9fa10 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:52:36.672807 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ea2499-ac6e-45d8-9373-62e297f9fa10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:52:36.682103 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ea2499-ac6e-45d8-9373-62e297f9fa10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:52:36.683500 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ea2499-ac6e-45d8-9373-62e297f9fa10 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:52:36.692779 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 958/1901] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:52:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:52:36.955627 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba63a2f5-c28b-4b9a-b170-1d8873686b9c None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:52:36.957387 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-ba63a2f5-c28b-4b9a-b170-1d8873686b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:52:36.969197 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba63a2f5-c28b-4b9a-b170-1d8873686b9c None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:52:36.974405 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fde4f2-e4bd-4593-901f-8a4dc6282739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:52:36.975116 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fde4f2-e4bd-4593-901f-8a4dc6282739 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:52:36.977714 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fde4f2-e4bd-4593-901f-8a4dc6282739 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:52:36.977714 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fde4f2-e4bd-4593-901f-8a4dc6282739 admin admin] PATH_INFO: `/v3/projects/4127b82b10e24d01aaa211bb70f236e9` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:52:36.977714 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fde4f2-e4bd-4593-901f-8a4dc6282739 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4127b82b10e24d01aaa211bb70f236e9)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:52:36.978834 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fde4f2-e4bd-4593-901f-8a4dc6282739 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:52:37.027848 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-a8fde4f2-e4bd-4593-901f-8a4dc6282739 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4127b82b10e24d01aaa211bb70f236e9', 'request_id': 'req-a8fde4f2-e4bd-4593-901f-8a4dc6282739'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:52:37.034956 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-a8fde4f2-e4bd-4593-901f-8a4dc6282739 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4127b82b10e24d01aaa211bb70f236e9', 'request_id': 'req-a8fde4f2-e4bd-4593-901f-8a4dc6282739'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:52:37.042484 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-a8fde4f2-e4bd-4593-901f-8a4dc6282739 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4127b82b10e24d01aaa211bb70f236e9', 'request_id': 'req-a8fde4f2-e4bd-4593-901f-8a4dc6282739'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:52:37.053657 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-a8fde4f2-e4bd-4593-901f-8a4dc6282739 admin admin] The token cache is being invalidate because project 4127b82b10e24d01aaa211bb70f236e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 08:52:37.053994 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-a8fde4f2-e4bd-4593-901f-8a4dc6282739 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:52:37.054626 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-a8fde4f2-e4bd-4593-901f-8a4dc6282739 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:52:37.056107 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 944/1902] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 08:52:36 2022] DELETE /identity/v3/projects/4127b82b10e24d01aaa211bb70f236e9 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 08:52:37.062067 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3acca6c-0da9-4cd2-9d51-ba68acd4accf None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:52:37.062430 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3acca6c-0da9-4cd2-9d51-ba68acd4accf None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:52:37.062776 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3acca6c-0da9-4cd2-9d51-ba68acd4accf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:52:37.113261 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-e3acca6c-0da9-4cd2-9d51-ba68acd4accf None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:52:37.114529 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-e3acca6c-0da9-4cd2-9d51-ba68acd4accf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:52:37.192785 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 959/1903] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:52:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:52:37.200856 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec0d29-c2bc-40cf-8d88-5edbd9e645cb None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:52:37.201206 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec0d29-c2bc-40cf-8d88-5edbd9e645cb None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:52:37.201545 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec0d29-c2bc-40cf-8d88-5edbd9e645cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:52:37.221634 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-0dec0d29-c2bc-40cf-8d88-5edbd9e645cb None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:52:37.222864 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-0dec0d29-c2bc-40cf-8d88-5edbd9e645cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:52:37.230851 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 945/1904] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:52:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:52:37.234807 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3acca6c-0da9-4cd2-9d51-ba68acd4accf None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:52:37.244539 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3acca6c-0da9-4cd2-9d51-ba68acd4accf None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:52:37.249208 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e31583c-8cd1-486f-9daa-91d42589b5e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:52:37.249851 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e31583c-8cd1-486f-9daa-91d42589b5e8 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:52:37.250139 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e31583c-8cd1-486f-9daa-91d42589b5e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:52:37.250420 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e31583c-8cd1-486f-9daa-91d42589b5e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:52:37.251101 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e31583c-8cd1-486f-9daa-91d42589b5e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:52:37.251907 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e31583c-8cd1-486f-9daa-91d42589b5e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:52:37.253508 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e31583c-8cd1-486f-9daa-91d42589b5e8 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:52:37.265321 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 960/1905] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 08:52:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:52:37.270232 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec0d29-c2bc-40cf-8d88-5edbd9e645cb None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:52:37.279252 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec0d29-c2bc-40cf-8d88-5edbd9e645cb None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:52:37.284609 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0876764-cb70-4494-92ce-a5b6e369ce79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:52:37.285193 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0876764-cb70-4494-92ce-a5b6e369ce79 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:52:37.285470 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0876764-cb70-4494-92ce-a5b6e369ce79 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:52:37.285779 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0876764-cb70-4494-92ce-a5b6e369ce79 admin admin] PATH_INFO: `/v3/projects` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:52:37.286401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0876764-cb70-4494-92ce-a5b6e369ce79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:52:37.288281 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0876764-cb70-4494-92ce-a5b6e369ce79 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:52:37.315687 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 946/1906] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 08:52:37 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:52:37.319751 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e31583c-8cd1-486f-9daa-91d42589b5e8 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:52:37.330047 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e31583c-8cd1-486f-9daa-91d42589b5e8 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:52:37.393384 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09625427-a8a8-420a-95c4-0906bdf7d2ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:52:37.398370 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09625427-a8a8-420a-95c4-0906bdf7d2ac admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:52:37.398370 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09625427-a8a8-420a-95c4-0906bdf7d2ac admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:52:37.398370 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09625427-a8a8-420a-95c4-0906bdf7d2ac admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:52:37.398370 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09625427-a8a8-420a-95c4-0906bdf7d2ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:52:37.398370 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09625427-a8a8-420a-95c4-0906bdf7d2ac admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:52:37.427151 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 961/1907] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:52:37 2022] POST /identity/v3/users => generated 287 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:52:37.431073 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0876764-cb70-4494-92ce-a5b6e369ce79 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:52:37.441536 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0876764-cb70-4494-92ce-a5b6e369ce79 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:52:37.446235 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9874250-1d93-4f15-b12a-1c8fdbefb585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:52:37.446841 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9874250-1d93-4f15-b12a-1c8fdbefb585 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:52:37.447125 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9874250-1d93-4f15-b12a-1c8fdbefb585 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:52:37.447403 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9874250-1d93-4f15-b12a-1c8fdbefb585 admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:52:37.448045 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9874250-1d93-4f15-b12a-1c8fdbefb585 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:52:37.449694 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9874250-1d93-4f15-b12a-1c8fdbefb585 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:52:37.461703 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 947/1908] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:52:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:52:37.465739 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09625427-a8a8-420a-95c4-0906bdf7d2ac admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:52:37.479122 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09625427-a8a8-420a-95c4-0906bdf7d2ac admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:52:37.483836 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce337a59-cd8b-43f8-8741-1b28860eec0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:52:37.484455 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce337a59-cd8b-43f8-8741-1b28860eec0f admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:52:37.484733 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce337a59-cd8b-43f8-8741-1b28860eec0f admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:52:37.485010 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce337a59-cd8b-43f8-8741-1b28860eec0f admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:52:37.485867 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce337a59-cd8b-43f8-8741-1b28860eec0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:52:37.487521 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce337a59-cd8b-43f8-8741-1b28860eec0f admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:52:37.498723 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 962/1909] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:52:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:52:37.502693 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9874250-1d93-4f15-b12a-1c8fdbefb585 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:52:37.512559 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9874250-1d93-4f15-b12a-1c8fdbefb585 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:52:37.518052 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d49830-b5cf-4a11-8a7b-1b636feb4c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:52:37.518717 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d49830-b5cf-4a11-8a7b-1b636feb4c69 admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:52:37.518998 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d49830-b5cf-4a11-8a7b-1b636feb4c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:52:37.519274 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d49830-b5cf-4a11-8a7b-1b636feb4c69 admin admin] PATH_INFO: `/v3/projects/2a5a53a6e11a453b95393ebf81044c98/users/114020057c0b4093a32231ef5212b2c9/roles/3248f0268738489b883041851d681bb7` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:52:37.540424 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d49830-b5cf-4a11-8a7b-1b636feb4c69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a5a53a6e11a453b95393ebf81044c98, user_id=114020057c0b4093a32231ef5212b2c9, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:52:37.543109 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d49830-b5cf-4a11-8a7b-1b636feb4c69 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:52:37.545242 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:52:37.559807 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 948/1910] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:52:37 2022] PUT /identity/v3/projects/2a5a53a6e11a453b95393ebf81044c98/users/114020057c0b4093a32231ef5212b2c9/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:52:37.571340 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847862e8-3eeb-40d4-86e3-f2c945933346 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:52:37.571630 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847862e8-3eeb-40d4-86e3-f2c945933346 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:52:37.571907 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847862e8-3eeb-40d4-86e3-f2c945933346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:52:37.585570 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:52:37.600925 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-847862e8-3eeb-40d4-86e3-f2c945933346 None None] MFA Rules not processed for user `114020057c0b4093a32231ef5212b2c9`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:52:37.602189 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-847862e8-3eeb-40d4-86e3-f2c945933346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:52:37.665390 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 963/1911] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:52:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:52:37.676439 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d49830-b5cf-4a11-8a7b-1b636feb4c69 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:52:37.678155 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-a5d49830-b5cf-4a11-8a7b-1b636feb4c69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:52:37.732930 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d49830-b5cf-4a11-8a7b-1b636feb4c69 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:52:37.737056 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79e54be-b5d9-49a0-a137-948ae79eb600 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:52:37.737647 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e54be-b5d9-49a0-a137-948ae79eb600 service cinder] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:52:37.737954 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e54be-b5d9-49a0-a137-948ae79eb600 service cinder] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:52:37.738224 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e54be-b5d9-49a0-a137-948ae79eb600 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:52:37.746205 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79e54be-b5d9-49a0-a137-948ae79eb600 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:52:37.747622 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79e54be-b5d9-49a0-a137-948ae79eb600 service cinder] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:52:37.756650 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 949/1912] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:52:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:52:46.702747 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847862e8-3eeb-40d4-86e3-f2c945933346 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:52:46.705276 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-847862e8-3eeb-40d4-86e3-f2c945933346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:52:46.893366 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79e54be-b5d9-49a0-a137-948ae79eb600 service cinder] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:52:46.896242 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-b79e54be-b5d9-49a0-a137-948ae79eb600 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:52:46.932057 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847862e8-3eeb-40d4-86e3-f2c945933346 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:52:46.957955 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b32b38-fd98-41fc-91e4-2a18d305ce86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0a8af5cadc4cc1ab73e8eb2a20bb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:52:46.957955 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b32b38-fd98-41fc-91e4-2a18d305ce86 service nova] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:52:46.957955 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b32b38-fd98-41fc-91e4-2a18d305ce86 service nova] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:52:46.957955 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b32b38-fd98-41fc-91e4-2a18d305ce86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:52:46.965263 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-48b32b38-fd98-41fc-91e4-2a18d305ce86 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:52:47.005504 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b32b38-fd98-41fc-91e4-2a18d305ce86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:52:47.007156 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b32b38-fd98-41fc-91e4-2a18d305ce86 service nova] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:52:47.034265 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79e54be-b5d9-49a0-a137-948ae79eb600 service cinder] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:52:47.040182 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 964/1913] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:52:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:52:47.040815 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1336ac6-9ef6-414a-a8d3-8f65278c5af2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38986ca2612c4fc59d960cd693834f9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:52:47.041630 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1336ac6-9ef6-414a-a8d3-8f65278c5af2 service placement] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:52:47.041939 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1336ac6-9ef6-414a-a8d3-8f65278c5af2 service placement] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:52:47.042222 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1336ac6-9ef6-414a-a8d3-8f65278c5af2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:52:47.044130 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-a1336ac6-9ef6-414a-a8d3-8f65278c5af2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:52:47.059125 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1336ac6-9ef6-414a-a8d3-8f65278c5af2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:52:47.061246 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1336ac6-9ef6-414a-a8d3-8f65278c5af2 service placement] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:52:47.073505 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 950/1914] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:52:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:52:51.302317 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2675d259-0b0d-440c-bc9b-f0488af8c264 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:52:51.302839 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2675d259-0b0d-440c-bc9b-f0488af8c264 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:52:51.302958 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2675d259-0b0d-440c-bc9b-f0488af8c264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:52:51.321830 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-2675d259-0b0d-440c-bc9b-f0488af8c264 None None] MFA Rules not processed for user `060d1fae9b0b488da1f7553cabd97d28`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:52:51.322955 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-2675d259-0b0d-440c-bc9b-f0488af8c264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:52:51.369315 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:52:51.377578 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 965/1915] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:52:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:52:51.675618 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1336ac6-9ef6-414a-a8d3-8f65278c5af2 service placement] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:52:51.677120 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-a1336ac6-9ef6-414a-a8d3-8f65278c5af2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:52:51.751893 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:52:51.757371 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1336ac6-9ef6-414a-a8d3-8f65278c5af2 service placement] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:52:51.761498 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944bfba5-3476-4e30-8399-20ee983d98da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:52:51.762152 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944bfba5-3476-4e30-8399-20ee983d98da service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:52:51.762432 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944bfba5-3476-4e30-8399-20ee983d98da service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:52:51.762709 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944bfba5-3476-4e30-8399-20ee983d98da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:52:51.764485 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-944bfba5-3476-4e30-8399-20ee983d98da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:52:51.774237 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944bfba5-3476-4e30-8399-20ee983d98da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:52:51.775678 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944bfba5-3476-4e30-8399-20ee983d98da service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:52:51.784354 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 951/1916] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:52:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:52:54.179661 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2675d259-0b0d-440c-bc9b-f0488af8c264 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:52:54.181148 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-2675d259-0b0d-440c-bc9b-f0488af8c264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:52:54.196112 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2675d259-0b0d-440c-bc9b-f0488af8c264 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:52:54.200972 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626d7b09-ba80-42b9-8db6-61aa4b4d237a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38986ca2612c4fc59d960cd693834f9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:52:54.201566 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626d7b09-ba80-42b9-8db6-61aa4b4d237a service placement] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:52:54.201880 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626d7b09-ba80-42b9-8db6-61aa4b4d237a service placement] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:52:54.202157 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626d7b09-ba80-42b9-8db6-61aa4b4d237a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:52:54.203799 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-626d7b09-ba80-42b9-8db6-61aa4b4d237a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:52:54.214658 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626d7b09-ba80-42b9-8db6-61aa4b4d237a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:52:54.216226 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626d7b09-ba80-42b9-8db6-61aa4b4d237a service placement] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:52:54.226292 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 966/1917] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:52:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:53:01.780343 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=5 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:53:03.461721 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944bfba5-3476-4e30-8399-20ee983d98da service neutron] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:53:03.476166 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944bfba5-3476-4e30-8399-20ee983d98da service neutron] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:53:03.481624 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6075a9d7-5a9b-4833-a472-3bafff7b19f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0a8af5cadc4cc1ab73e8eb2a20bb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:53:03.482256 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6075a9d7-5a9b-4833-a472-3bafff7b19f8 service nova] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:53:03.482539 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6075a9d7-5a9b-4833-a472-3bafff7b19f8 service nova] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:53:03.482813 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6075a9d7-5a9b-4833-a472-3bafff7b19f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:53:03.484518 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-6075a9d7-5a9b-4833-a472-3bafff7b19f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:53:03.497371 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6075a9d7-5a9b-4833-a472-3bafff7b19f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:53:03.499004 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6075a9d7-5a9b-4833-a472-3bafff7b19f8 service nova] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:53:03.510105 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 952/1918] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:53:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:53:04.222206 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=6 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:53:13.505614 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=6 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:54:04.121985 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca8da32-1822-41c5-a596-6dad11d3a95d None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:54:04.122490 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca8da32-1822-41c5-a596-6dad11d3a95d None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:54:04.122626 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca8da32-1822-41c5-a596-6dad11d3a95d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:54:04.151674 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-4ca8da32-1822-41c5-a596-6dad11d3a95d None None] MFA Rules not processed for user `c5b2a0e623c348ad94d4820080677be6`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:54:04.152849 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-4ca8da32-1822-41c5-a596-6dad11d3a95d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:54:04.206484 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 967/1919] 10.222.0.30 () {58 vars in 967 bytes} [Sat Oct 15 08:54:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:54:04.259261 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6075a9d7-5a9b-4833-a472-3bafff7b19f8 service nova] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:54:04.260756 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-6075a9d7-5a9b-4833-a472-3bafff7b19f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:54:04.327399 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6075a9d7-5a9b-4833-a472-3bafff7b19f8 service nova] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:54:04.332424 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068a8cac-64d1-43e4-a8e7-d49d3ad655c1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e3009e20e4c49e5a74f7fd793b2c329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:54:04.333082 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068a8cac-64d1-43e4-a8e7-d49d3ad655c1 service swift] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:54:04.333387 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068a8cac-64d1-43e4-a8e7-d49d3ad655c1 service swift] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:54:04.333664 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068a8cac-64d1-43e4-a8e7-d49d3ad655c1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:54:04.341628 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068a8cac-64d1-43e4-a8e7-d49d3ad655c1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:54:04.343218 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068a8cac-64d1-43e4-a8e7-d49d3ad655c1 service swift] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:54:04.351111 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 953/1920] 10.222.0.30 () {60 vars in 1384 bytes} [Sat Oct 15 08:54:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 08:54:05.609786 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e81f4-625a-4115-9599-f661293154bb None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:54:05.610329 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e81f4-625a-4115-9599-f661293154bb None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:54:05.610427 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e81f4-625a-4115-9599-f661293154bb None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:54:05.612393 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 968/1921] 10.222.0.30 () {54 vars in 912 bytes} [Sat Oct 15 08:54:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 08:54:05.618550 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5cf58b-45f4-4e97-8bae-4f33b8452aff None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:54:05.619023 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5cf58b-45f4-4e97-8bae-4f33b8452aff None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:54:05.619165 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5cf58b-45f4-4e97-8bae-4f33b8452aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:54:05.664532 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-6b5cf58b-45f4-4e97-8bae-4f33b8452aff None None] MFA Rules not processed for user `38986ca2612c4fc59d960cd693834f9b`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:54:05.665828 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-6b5cf58b-45f4-4e97-8bae-4f33b8452aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:54:05.675163 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 954/1922] 10.222.0.30 () {58 vars in 1009 bytes} [Sat Oct 15 08:54:05 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:54:05.698842 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6e81f4-625a-4115-9599-f661293154bb None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:54:05.714235 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6e81f4-625a-4115-9599-f661293154bb None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:54:05.719167 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdb78b9-4bb3-4a39-9dca-72e357013915 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38986ca2612c4fc59d960cd693834f9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:54:05.719830 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdb78b9-4bb3-4a39-9dca-72e357013915 service placement] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:54:05.720110 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdb78b9-4bb3-4a39-9dca-72e357013915 service placement] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:54:05.720387 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdb78b9-4bb3-4a39-9dca-72e357013915 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:54:05.729539 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdb78b9-4bb3-4a39-9dca-72e357013915 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:54:05.731246 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdb78b9-4bb3-4a39-9dca-72e357013915 service placement] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:54:05.740891 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 969/1923] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:54:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:54:07.911180 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9f1b19-deee-4e7d-926f-3f2a5ab6fe37 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:54:07.911674 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9f1b19-deee-4e7d-926f-3f2a5ab6fe37 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:54:07.911780 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9f1b19-deee-4e7d-926f-3f2a5ab6fe37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:54:07.930801 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-7e9f1b19-deee-4e7d-926f-3f2a5ab6fe37 None None] MFA Rules not processed for user `388ca829d87348bd9b524fd9a80e578b`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:54:07.932007 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-7e9f1b19-deee-4e7d-926f-3f2a5ab6fe37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:54:07.985347 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 955/1924] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:54:07 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:54:15.736795 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=35 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:54:17.978769 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=66 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:54:33.688429 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdb78b9-4bb3-4a39-9dca-72e357013915 service placement] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:54:33.700745 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdb78b9-4bb3-4a39-9dca-72e357013915 service placement] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:54:33.706345 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9be6cd-20b2-4eea-877d-d22eecaa5b39 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0a8af5cadc4cc1ab73e8eb2a20bb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:54:33.706949 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9be6cd-20b2-4eea-877d-d22eecaa5b39 service nova] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:54:33.707232 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9be6cd-20b2-4eea-877d-d22eecaa5b39 service nova] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:54:33.707507 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9be6cd-20b2-4eea-877d-d22eecaa5b39 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:54:33.709915 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-6c9be6cd-20b2-4eea-877d-d22eecaa5b39 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:54:33.788976 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9be6cd-20b2-4eea-877d-d22eecaa5b39 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:54:33.790605 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9be6cd-20b2-4eea-877d-d22eecaa5b39 service nova] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:54:33.808584 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 970/1925] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:54:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:54:41.008630 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9f1b19-deee-4e7d-926f-3f2a5ab6fe37 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:54:41.032440 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9f1b19-deee-4e7d-926f-3f2a5ab6fe37 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:54:41.036832 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14807fa-1a31-4d17-888a-aafff346af8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:54:41.037512 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14807fa-1a31-4d17-888a-aafff346af8e service cinder] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:54:41.039099 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14807fa-1a31-4d17-888a-aafff346af8e service cinder] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:54:41.039286 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14807fa-1a31-4d17-888a-aafff346af8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:54:41.040917 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-f14807fa-1a31-4d17-888a-aafff346af8e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:54:41.125030 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14807fa-1a31-4d17-888a-aafff346af8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:54:41.126507 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14807fa-1a31-4d17-888a-aafff346af8e service cinder] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:54:41.145807 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 956/1926] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:54:41 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:54:43.805799 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=41 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:54:46.210513 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9be6cd-20b2-4eea-877d-d22eecaa5b39 service nova] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:54:46.220042 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9be6cd-20b2-4eea-877d-d22eecaa5b39 service nova] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:54:46.223825 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8fe3a7-d6a6-4aa5-85f4-e24e551cf48d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:54:46.224428 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8fe3a7-d6a6-4aa5-85f4-e24e551cf48d service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:54:46.224710 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8fe3a7-d6a6-4aa5-85f4-e24e551cf48d service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:54:46.224987 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8fe3a7-d6a6-4aa5-85f4-e24e551cf48d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:54:46.226774 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-6c8fe3a7-d6a6-4aa5-85f4-e24e551cf48d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:54:46.261388 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8fe3a7-d6a6-4aa5-85f4-e24e551cf48d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:54:46.262893 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8fe3a7-d6a6-4aa5-85f4-e24e551cf48d service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:54:46.280743 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 971/1927] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:54:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:54:51.142906 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=45 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:54:53.919533 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14807fa-1a31-4d17-888a-aafff346af8e service cinder] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:54:53.931461 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14807fa-1a31-4d17-888a-aafff346af8e service cinder] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:54:53.936725 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14bbd18-0cc2-48e8-8f7c-8b8a896b5fa9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0a8af5cadc4cc1ab73e8eb2a20bb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:54:53.937318 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14bbd18-0cc2-48e8-8f7c-8b8a896b5fa9 service nova] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:54:53.937595 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14bbd18-0cc2-48e8-8f7c-8b8a896b5fa9 service nova] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:54:53.937897 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14bbd18-0cc2-48e8-8f7c-8b8a896b5fa9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:54:53.939561 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-f14bbd18-0cc2-48e8-8f7c-8b8a896b5fa9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:54:53.971132 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14bbd18-0cc2-48e8-8f7c-8b8a896b5fa9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:54:53.972838 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14bbd18-0cc2-48e8-8f7c-8b8a896b5fa9 service nova] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:54:53.991018 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 957/1928] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:54:53 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:54:56.278120 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=19 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:55:03.988344 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=19 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:55:48.697507 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8fe3a7-d6a6-4aa5-85f4-e24e551cf48d service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:55:48.706860 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8fe3a7-d6a6-4aa5-85f4-e24e551cf48d service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:55:48.710367 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3bc25c-cbc8-4460-a925-53248042c6bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:55:48.710957 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3bc25c-cbc8-4460-a925-53248042c6bc service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:55:48.711237 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3bc25c-cbc8-4460-a925-53248042c6bc service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:55:48.711532 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3bc25c-cbc8-4460-a925-53248042c6bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:55:48.713208 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-fb3bc25c-cbc8-4460-a925-53248042c6bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:55:48.722274 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3bc25c-cbc8-4460-a925-53248042c6bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:55:48.723680 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3bc25c-cbc8-4460-a925-53248042c6bc service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:55:48.732264 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 972/1929] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:55:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:55:58.728904 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=6 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:56:02.095405 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14bbd18-0cc2-48e8-8f7c-8b8a896b5fa9 service nova] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:56:02.096814 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-f14bbd18-0cc2-48e8-8f7c-8b8a896b5fa9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:56:02.120285 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14bbd18-0cc2-48e8-8f7c-8b8a896b5fa9 service nova] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:56:02.123829 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21c78a5-1d5d-446a-9ab3-ea9f7b117a0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:56:02.124424 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c78a5-1d5d-446a-9ab3-ea9f7b117a0f service cinder] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:56:02.124701 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c78a5-1d5d-446a-9ab3-ea9f7b117a0f service cinder] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:56:02.124969 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c78a5-1d5d-446a-9ab3-ea9f7b117a0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:56:02.126649 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-c21c78a5-1d5d-446a-9ab3-ea9f7b117a0f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:56:02.205646 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21c78a5-1d5d-446a-9ab3-ea9f7b117a0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:56:02.207102 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21c78a5-1d5d-446a-9ab3-ea9f7b117a0f service cinder] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:56:02.224684 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 958/1930] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:56:02 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:56:12.221915 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=46 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:56:15.056606 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3bc25c-cbc8-4460-a925-53248042c6bc service neutron] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:56:15.066053 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3bc25c-cbc8-4460-a925-53248042c6bc service neutron] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:56:15.069582 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7440c3f-8779-4962-8dd8-8de1d053dc52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:56:15.070208 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7440c3f-8779-4962-8dd8-8de1d053dc52 service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:56:15.070493 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7440c3f-8779-4962-8dd8-8de1d053dc52 service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:56:15.070772 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7440c3f-8779-4962-8dd8-8de1d053dc52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:56:15.073028 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-b7440c3f-8779-4962-8dd8-8de1d053dc52 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:56:15.152688 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7440c3f-8779-4962-8dd8-8de1d053dc52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:56:15.154148 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7440c3f-8779-4962-8dd8-8de1d053dc52 service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:56:15.172034 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 973/1931] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:56:15 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:56:25.169462 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=41 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:57:37.691726 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21c78a5-1d5d-446a-9ab3-ea9f7b117a0f service cinder] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:57:37.700994 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21c78a5-1d5d-446a-9ab3-ea9f7b117a0f service cinder] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:57:37.704587 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2f3a8b-09f6-45ae-a0b8-2e4f27f76d45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:57:37.705187 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f3a8b-09f6-45ae-a0b8-2e4f27f76d45 service cinder] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:57:37.705463 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f3a8b-09f6-45ae-a0b8-2e4f27f76d45 service cinder] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:57:37.705756 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f3a8b-09f6-45ae-a0b8-2e4f27f76d45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:57:37.714510 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2f3a8b-09f6-45ae-a0b8-2e4f27f76d45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:57:37.715923 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2f3a8b-09f6-45ae-a0b8-2e4f27f76d45 service cinder] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:57:37.724906 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 959/1932] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:57:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:57:47.720627 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=6 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:58:38.150671 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7440c3f-8779-4962-8dd8-8de1d053dc52 service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:58:38.211736 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7440c3f-8779-4962-8dd8-8de1d053dc52 service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:58:38.216188 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a65314f-bbae-433d-9a35-a1e196b0e7f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:58:38.216814 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a65314f-bbae-433d-9a35-a1e196b0e7f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:58:38.217095 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a65314f-bbae-433d-9a35-a1e196b0e7f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:58:38.217367 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a65314f-bbae-433d-9a35-a1e196b0e7f3 admin admin] PATH_INFO: `/v3/users/114020057c0b4093a32231ef5212b2c9` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:58:38.218504 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a65314f-bbae-433d-9a35-a1e196b0e7f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=114020057c0b4093a32231ef5212b2c9)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:58:38.220403 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a65314f-bbae-433d-9a35-a1e196b0e7f3 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:58:38.264489 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8a65314f-bbae-433d-9a35-a1e196b0e7f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '114020057c0b4093a32231ef5212b2c9'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:58:38.265145 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8a65314f-bbae-433d-9a35-a1e196b0e7f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '114020057c0b4093a32231ef5212b2c9'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:58:38.271871 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8a65314f-bbae-433d-9a35-a1e196b0e7f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '114020057c0b4093a32231ef5212b2c9'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:58:38.284291 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8a65314f-bbae-433d-9a35-a1e196b0e7f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '114020057c0b4093a32231ef5212b2c9'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:58:38.292975 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8a65314f-bbae-433d-9a35-a1e196b0e7f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '114020057c0b4093a32231ef5212b2c9'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:58:38.298836 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 974/1933] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 08:58:38 2022] DELETE /identity/v3/users/114020057c0b4093a32231ef5212b2c9 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:58:38.304511 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cbf493-a390-4c3e-9174-9b768fa72454 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:58:38.305014 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cbf493-a390-4c3e-9174-9b768fa72454 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:58:38.305122 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cbf493-a390-4c3e-9174-9b768fa72454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:58:38.324452 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-e6cbf493-a390-4c3e-9174-9b768fa72454 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:58:38.325714 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-e6cbf493-a390-4c3e-9174-9b768fa72454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:58:38.387620 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 960/1934] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:58:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:58:38.398543 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a65314f-bbae-433d-9a35-a1e196b0e7f3 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:58:38.400043 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-8a65314f-bbae-433d-9a35-a1e196b0e7f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:58:38.457560 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a65314f-bbae-433d-9a35-a1e196b0e7f3 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:58:38.461357 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa75081d-ff1d-465d-8830-6c23fd6ad5f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:58:38.462006 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa75081d-ff1d-465d-8830-6c23fd6ad5f8 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:58:38.462291 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa75081d-ff1d-465d-8830-6c23fd6ad5f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:58:38.462569 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa75081d-ff1d-465d-8830-6c23fd6ad5f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:58:38.471470 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa75081d-ff1d-465d-8830-6c23fd6ad5f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:58:38.472915 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa75081d-ff1d-465d-8830-6c23fd6ad5f8 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:58:38.482541 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 975/1935] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:58:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 08:58:38.740926 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cbf493-a390-4c3e-9174-9b768fa72454 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:58:38.742386 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-e6cbf493-a390-4c3e-9174-9b768fa72454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:58:38.754194 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cbf493-a390-4c3e-9174-9b768fa72454 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:58:38.759633 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21effa18-17bf-4bca-9661-b3b1e75023df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:58:38.760269 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21effa18-17bf-4bca-9661-b3b1e75023df admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:58:38.760552 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21effa18-17bf-4bca-9661-b3b1e75023df admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:58:38.760829 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21effa18-17bf-4bca-9661-b3b1e75023df admin admin] PATH_INFO: `/v3/projects/2a5a53a6e11a453b95393ebf81044c98` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:58:38.761991 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21effa18-17bf-4bca-9661-b3b1e75023df admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a5a53a6e11a453b95393ebf81044c98)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:58:38.763917 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21effa18-17bf-4bca-9661-b3b1e75023df admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:58:38.813169 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-21effa18-17bf-4bca-9661-b3b1e75023df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a5a53a6e11a453b95393ebf81044c98', 'request_id': 'req-21effa18-17bf-4bca-9661-b3b1e75023df'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:58:38.819477 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-21effa18-17bf-4bca-9661-b3b1e75023df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a5a53a6e11a453b95393ebf81044c98', 'request_id': 'req-21effa18-17bf-4bca-9661-b3b1e75023df'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:58:38.826882 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-21effa18-17bf-4bca-9661-b3b1e75023df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a5a53a6e11a453b95393ebf81044c98', 'request_id': 'req-21effa18-17bf-4bca-9661-b3b1e75023df'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:58:38.837184 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-21effa18-17bf-4bca-9661-b3b1e75023df admin admin] The token cache is being invalidate because project 2a5a53a6e11a453b95393ebf81044c98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 08:58:38.837481 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-21effa18-17bf-4bca-9661-b3b1e75023df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:58:38.838133 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-21effa18-17bf-4bca-9661-b3b1e75023df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:58:38.839619 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 961/1936] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 08:58:38 2022] DELETE /identity/v3/projects/2a5a53a6e11a453b95393ebf81044c98 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:58:38.845762 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2204b008-b6e9-4bb4-af32-8416385485e4 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:58:38.846049 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2204b008-b6e9-4bb4-af32-8416385485e4 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:58:38.846329 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2204b008-b6e9-4bb4-af32-8416385485e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:58:38.866289 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-2204b008-b6e9-4bb4-af32-8416385485e4 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:58:38.867481 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-2204b008-b6e9-4bb4-af32-8416385485e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:58:38.883767 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:58:38.926908 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 976/1937] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:58:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:58:38.934927 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b87be1-6ba1-4e68-a0cd-d75143f96cf2 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:58:38.935213 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b87be1-6ba1-4e68-a0cd-d75143f96cf2 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:58:38.935490 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b87be1-6ba1-4e68-a0cd-d75143f96cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:58:38.955042 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-55b87be1-6ba1-4e68-a0cd-d75143f96cf2 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:58:38.956228 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-55b87be1-6ba1-4e68-a0cd-d75143f96cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:58:38.964589 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 962/1938] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:58:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:58:38.968771 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2204b008-b6e9-4bb4-af32-8416385485e4 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:58:38.979057 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2204b008-b6e9-4bb4-af32-8416385485e4 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:58:38.983581 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb34e303-5e06-45df-b8ea-96fb9b6d8688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:58:38.984184 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb34e303-5e06-45df-b8ea-96fb9b6d8688 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:58:38.984467 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb34e303-5e06-45df-b8ea-96fb9b6d8688 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:58:38.984744 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb34e303-5e06-45df-b8ea-96fb9b6d8688 admin admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:58:38.985390 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb34e303-5e06-45df-b8ea-96fb9b6d8688 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 08:58:38.985722 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb34e303-5e06-45df-b8ea-96fb9b6d8688 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:58:38.987375 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb34e303-5e06-45df-b8ea-96fb9b6d8688 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:58:39.001761 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 977/1939] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 08:58:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 08:58:39.006122 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b87be1-6ba1-4e68-a0cd-d75143f96cf2 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:58:39.015530 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b87be1-6ba1-4e68-a0cd-d75143f96cf2 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:58:39.020246 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21cb353-1d1f-48d6-a936-d27f14c380dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:58:39.020878 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21cb353-1d1f-48d6-a936-d27f14c380dc admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:58:39.021179 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21cb353-1d1f-48d6-a936-d27f14c380dc admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:58:39.021439 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21cb353-1d1f-48d6-a936-d27f14c380dc admin admin] PATH_INFO: `/v3/projects` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:58:39.022116 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21cb353-1d1f-48d6-a936-d27f14c380dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:58:39.024086 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21cb353-1d1f-48d6-a936-d27f14c380dc admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:58:39.064959 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 963/1940] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 08:58:39 2022] POST /identity/v3/projects => generated 369 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 08:58:39.066237 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb34e303-5e06-45df-b8ea-96fb9b6d8688 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:58:39.076759 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb34e303-5e06-45df-b8ea-96fb9b6d8688 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:58:39.134834 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a2a86-36ec-4ffd-a95a-e4a9e5c5433e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:58:39.135470 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a2a86-36ec-4ffd-a95a-e4a9e5c5433e admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:58:39.135754 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a2a86-36ec-4ffd-a95a-e4a9e5c5433e admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:58:39.136034 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a2a86-36ec-4ffd-a95a-e4a9e5c5433e admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:58:39.136666 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9a2a86-36ec-4ffd-a95a-e4a9e5c5433e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:58:39.138610 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9a2a86-36ec-4ffd-a95a-e4a9e5c5433e admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:58:39.164972 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 978/1941] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 08:58:39 2022] POST /identity/v3/users => generated 285 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 08:58:39.166161 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21cb353-1d1f-48d6-a936-d27f14c380dc admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:58:39.176287 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21cb353-1d1f-48d6-a936-d27f14c380dc admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:58:39.181529 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6918f813-5665-413c-92dd-3a6dcc0476ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:58:39.182096 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6918f813-5665-413c-92dd-3a6dcc0476ae admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:58:39.182381 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6918f813-5665-413c-92dd-3a6dcc0476ae admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:58:39.182655 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6918f813-5665-413c-92dd-3a6dcc0476ae admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:58:39.183295 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6918f813-5665-413c-92dd-3a6dcc0476ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:58:39.184890 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6918f813-5665-413c-92dd-3a6dcc0476ae admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:58:39.194903 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 964/1942] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:58:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 08:58:39.198915 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a2a86-36ec-4ffd-a95a-e4a9e5c5433e admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:58:39.209193 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a2a86-36ec-4ffd-a95a-e4a9e5c5433e admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:58:39.213803 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af8ad2d-2879-4d3d-a9b0-dcc7cc74bac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:58:39.214396 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af8ad2d-2879-4d3d-a9b0-dcc7cc74bac4 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:58:39.214681 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af8ad2d-2879-4d3d-a9b0-dcc7cc74bac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:58:39.214955 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af8ad2d-2879-4d3d-a9b0-dcc7cc74bac4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:58:39.215593 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af8ad2d-2879-4d3d-a9b0-dcc7cc74bac4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:58:39.217175 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af8ad2d-2879-4d3d-a9b0-dcc7cc74bac4 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:58:39.228334 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 979/1943] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 08:58:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 08:58:39.232367 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6918f813-5665-413c-92dd-3a6dcc0476ae admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:58:39.242088 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6918f813-5665-413c-92dd-3a6dcc0476ae admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:58:39.246581 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca57f90-fb85-410e-85bc-a32ccea2b649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:58:39.247218 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca57f90-fb85-410e-85bc-a32ccea2b649 admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:58:39.247496 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca57f90-fb85-410e-85bc-a32ccea2b649 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:58:39.247769 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca57f90-fb85-410e-85bc-a32ccea2b649 admin admin] PATH_INFO: `/v3/projects/41425b7af0c54491be7d15b42790dee0/users/ca46ccf45ef54624880d68d3f7091617/roles/3248f0268738489b883041851d681bb7` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:58:39.262177 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca57f90-fb85-410e-85bc-a32ccea2b649 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41425b7af0c54491be7d15b42790dee0, user_id=ca46ccf45ef54624880d68d3f7091617, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:58:39.264781 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca57f90-fb85-410e-85bc-a32ccea2b649 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:58:39.278001 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 965/1944] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 08:58:39 2022] PUT /identity/v3/projects/41425b7af0c54491be7d15b42790dee0/users/ca46ccf45ef54624880d68d3f7091617/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:58:39.288744 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe49fd9-b021-47ed-95f5-cf1b1cb556bd None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:58:39.289084 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe49fd9-b021-47ed-95f5-cf1b1cb556bd None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:58:39.289362 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe49fd9-b021-47ed-95f5-cf1b1cb556bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:58:39.316245 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-efe49fd9-b021-47ed-95f5-cf1b1cb556bd None None] MFA Rules not processed for user `ca46ccf45ef54624880d68d3f7091617`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:58:39.317463 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-efe49fd9-b021-47ed-95f5-cf1b1cb556bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:58:39.380533 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 980/1945] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:58:39 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:58:39.392728 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca57f90-fb85-410e-85bc-a32ccea2b649 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:58:39.395190 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-fca57f90-fb85-410e-85bc-a32ccea2b649 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:58:39.409371 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:58:39.476499 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca57f90-fb85-410e-85bc-a32ccea2b649 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:58:39.480031 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eeb948-c096-4e5f-a5ca-e6bb3088d53f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71692d05fd324679b1598feb52b3fe6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:58:39.480659 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eeb948-c096-4e5f-a5ca-e6bb3088d53f service glance] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:58:39.480937 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eeb948-c096-4e5f-a5ca-e6bb3088d53f service glance] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:58:39.481211 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eeb948-c096-4e5f-a5ca-e6bb3088d53f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:58:39.489276 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eeb948-c096-4e5f-a5ca-e6bb3088d53f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:58:39.490755 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eeb948-c096-4e5f-a5ca-e6bb3088d53f service glance] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:58:39.499225 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 966/1946] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:58:39 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:58:45.076668 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe49fd9-b021-47ed-95f5-cf1b1cb556bd None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:58:45.078114 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-efe49fd9-b021-47ed-95f5-cf1b1cb556bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:58:45.100484 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:58:45.152912 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe49fd9-b021-47ed-95f5-cf1b1cb556bd None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:58:45.157925 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cad5647-7b5b-4b1a-aa50-00f09732777d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38986ca2612c4fc59d960cd693834f9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:58:45.158547 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cad5647-7b5b-4b1a-aa50-00f09732777d service placement] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:58:45.158836 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cad5647-7b5b-4b1a-aa50-00f09732777d service placement] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:58:45.159120 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cad5647-7b5b-4b1a-aa50-00f09732777d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:58:45.161232 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-1cad5647-7b5b-4b1a-aa50-00f09732777d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:58:45.174594 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cad5647-7b5b-4b1a-aa50-00f09732777d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:58:45.176211 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cad5647-7b5b-4b1a-aa50-00f09732777d service placement] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:58:45.186736 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 981/1947] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:58:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:58:49.495306 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=30 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:58:54.045525 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eeb948-c096-4e5f-a5ca-e6bb3088d53f service glance] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:58:54.047652 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-41eeb948-c096-4e5f-a5ca-e6bb3088d53f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:58:54.105193 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eeb948-c096-4e5f-a5ca-e6bb3088d53f service glance] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:58:54.108721 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c28ea84-8783-4d06-9ddc-ae36cb693280 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:58:54.109322 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c28ea84-8783-4d06-9ddc-ae36cb693280 service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:58:54.109598 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c28ea84-8783-4d06-9ddc-ae36cb693280 service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:58:54.109898 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c28ea84-8783-4d06-9ddc-ae36cb693280 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:58:54.111585 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-8c28ea84-8783-4d06-9ddc-ae36cb693280 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:58:54.196954 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c28ea84-8783-4d06-9ddc-ae36cb693280 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:58:54.198435 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c28ea84-8783-4d06-9ddc-ae36cb693280 service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:58:54.209232 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 967/1948] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:58:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:58:55.182708 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=24 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:58:55.388441 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cad5647-7b5b-4b1a-aa50-00f09732777d service placement] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:58:55.398659 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cad5647-7b5b-4b1a-aa50-00f09732777d service placement] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:58:55.402527 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa8f8ee-3573-4c61-ae71-fba02eb530e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:58:55.403130 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa8f8ee-3573-4c61-ae71-fba02eb530e1 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:58:55.403412 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa8f8ee-3573-4c61-ae71-fba02eb530e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:58:55.403687 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa8f8ee-3573-4c61-ae71-fba02eb530e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:58:55.405385 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-0aa8f8ee-3573-4c61-ae71-fba02eb530e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:58:55.418866 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa8f8ee-3573-4c61-ae71-fba02eb530e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:58:55.420302 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa8f8ee-3573-4c61-ae71-fba02eb530e1 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:58:55.430903 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 982/1949] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:58:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:58:56.112209 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c28ea84-8783-4d06-9ddc-ae36cb693280 service neutron] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:58:56.113628 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-8c28ea84-8783-4d06-9ddc-ae36cb693280 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:58:56.128237 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c28ea84-8783-4d06-9ddc-ae36cb693280 service neutron] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:58:56.133997 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6581c9-10bf-44e3-a3bf-81ac238e0ebf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0a8af5cadc4cc1ab73e8eb2a20bb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:58:56.134701 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6581c9-10bf-44e3-a3bf-81ac238e0ebf service nova] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:58:56.134989 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6581c9-10bf-44e3-a3bf-81ac238e0ebf service nova] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:58:56.135264 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6581c9-10bf-44e3-a3bf-81ac238e0ebf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:58:56.137009 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-7b6581c9-10bf-44e3-a3bf-81ac238e0ebf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:58:56.149273 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6581c9-10bf-44e3-a3bf-81ac238e0ebf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:58:56.150894 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6581c9-10bf-44e3-a3bf-81ac238e0ebf service nova] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:58:56.161779 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 968/1950] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:58:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:58:57.516074 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb03ab76-0460-4036-a85b-d13bf1540408 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:58:57.516506 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb03ab76-0460-4036-a85b-d13bf1540408 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:58:57.516690 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb03ab76-0460-4036-a85b-d13bf1540408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:58:57.566737 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-cb03ab76-0460-4036-a85b-d13bf1540408 None None] MFA Rules not processed for user `c5b2a0e623c348ad94d4820080677be6`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:58:57.567878 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-cb03ab76-0460-4036-a85b-d13bf1540408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:58:57.629957 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 983/1951] 10.222.0.30 () {58 vars in 967 bytes} [Sat Oct 15 08:58:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:58:57.677236 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6581c9-10bf-44e3-a3bf-81ac238e0ebf service nova] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:58:57.679309 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-7b6581c9-10bf-44e3-a3bf-81ac238e0ebf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:58:57.767003 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6581c9-10bf-44e3-a3bf-81ac238e0ebf service nova] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:58:57.772046 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a192441-ead8-4165-8784-87624aee26f0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e3009e20e4c49e5a74f7fd793b2c329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:58:57.772652 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a192441-ead8-4165-8784-87624aee26f0 service swift] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:58:57.772933 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a192441-ead8-4165-8784-87624aee26f0 service swift] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:58:57.773205 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a192441-ead8-4165-8784-87624aee26f0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:58:57.781553 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a192441-ead8-4165-8784-87624aee26f0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:58:57.783180 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a192441-ead8-4165-8784-87624aee26f0 service swift] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:58:57.787234 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:58:57.793968 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 969/1952] 10.222.0.30 () {60 vars in 1384 bytes} [Sat Oct 15 08:58:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 08:58:58.244466 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb03ab76-0460-4036-a85b-d13bf1540408 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:58:58.253998 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb03ab76-0460-4036-a85b-d13bf1540408 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:58:58.258172 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7598578-a931-4bb2-b7ba-71eaf75a0d3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:58:58.258812 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7598578-a931-4bb2-b7ba-71eaf75a0d3d service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:58:58.259094 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7598578-a931-4bb2-b7ba-71eaf75a0d3d service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:58:58.259370 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7598578-a931-4bb2-b7ba-71eaf75a0d3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:58:58.261116 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-c7598578-a931-4bb2-b7ba-71eaf75a0d3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:58:58.326956 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7598578-a931-4bb2-b7ba-71eaf75a0d3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:58:58.328296 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7598578-a931-4bb2-b7ba-71eaf75a0d3d service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:58:58.338452 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 984/1953] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:58:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:59:01.601661 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a192441-ead8-4165-8784-87624aee26f0 service swift] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:59:01.614493 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a192441-ead8-4165-8784-87624aee26f0 service swift] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:59:01.621164 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5670cee5-abfa-4848-8a18-22d4e5b42a89 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38986ca2612c4fc59d960cd693834f9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:59:01.626224 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5670cee5-abfa-4848-8a18-22d4e5b42a89 service placement] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:59:01.626689 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5670cee5-abfa-4848-8a18-22d4e5b42a89 service placement] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:59:01.627370 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5670cee5-abfa-4848-8a18-22d4e5b42a89 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:59:01.629635 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-5670cee5-abfa-4848-8a18-22d4e5b42a89 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:59:01.649127 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5670cee5-abfa-4848-8a18-22d4e5b42a89 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:59:01.650775 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5670cee5-abfa-4848-8a18-22d4e5b42a89 service placement] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:59:01.661463 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 970/1954] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:59:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:59:04.535120 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7598578-a931-4bb2-b7ba-71eaf75a0d3d service neutron] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:59:04.538623 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-c7598578-a931-4bb2-b7ba-71eaf75a0d3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:59:04.554331 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7598578-a931-4bb2-b7ba-71eaf75a0d3d service neutron] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:59:04.559628 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a133e9-1081-4d6b-b1c8-7dfdeccc1fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:59:04.560294 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a133e9-1081-4d6b-b1c8-7dfdeccc1fa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:59:04.560576 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a133e9-1081-4d6b-b1c8-7dfdeccc1fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:59:04.560852 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a133e9-1081-4d6b-b1c8-7dfdeccc1fa3 admin admin] PATH_INFO: `/v3/users/f05cbab4640048589b9509fd9e8f655a` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:59:04.562016 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a133e9-1081-4d6b-b1c8-7dfdeccc1fa3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f05cbab4640048589b9509fd9e8f655a)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:59:04.563981 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a133e9-1081-4d6b-b1c8-7dfdeccc1fa3 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:59:04.614660 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-c3a133e9-1081-4d6b-b1c8-7dfdeccc1fa3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f05cbab4640048589b9509fd9e8f655a'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:59:04.615256 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-c3a133e9-1081-4d6b-b1c8-7dfdeccc1fa3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f05cbab4640048589b9509fd9e8f655a'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:59:04.616760 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:59:04.624711 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-c3a133e9-1081-4d6b-b1c8-7dfdeccc1fa3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f05cbab4640048589b9509fd9e8f655a'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:59:04.639177 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-c3a133e9-1081-4d6b-b1c8-7dfdeccc1fa3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f05cbab4640048589b9509fd9e8f655a'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:59:04.648144 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-c3a133e9-1081-4d6b-b1c8-7dfdeccc1fa3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f05cbab4640048589b9509fd9e8f655a'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:59:04.654805 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 985/1955] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 08:59:04 2022] DELETE /identity/v3/users/f05cbab4640048589b9509fd9e8f655a => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:59:04.834802 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5670cee5-abfa-4848-8a18-22d4e5b42a89 service placement] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:59:04.848059 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5670cee5-abfa-4848-8a18-22d4e5b42a89 service placement] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:59:04.904405 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91528d4a-1701-4c75-b20c-ad0e2132048c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:59:04.904967 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91528d4a-1701-4c75-b20c-ad0e2132048c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:59:04.905248 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91528d4a-1701-4c75-b20c-ad0e2132048c admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:59:04.905522 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91528d4a-1701-4c75-b20c-ad0e2132048c admin admin] PATH_INFO: `/v3/projects/678aa241e39f4bea9fa9e73e82afbd0b` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:59:04.906917 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91528d4a-1701-4c75-b20c-ad0e2132048c admin admin] RBAC: Authorizing `identity:delete_project(project_id=678aa241e39f4bea9fa9e73e82afbd0b)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:59:04.908845 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91528d4a-1701-4c75-b20c-ad0e2132048c admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:59:04.954965 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-91528d4a-1701-4c75-b20c-ad0e2132048c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '678aa241e39f4bea9fa9e73e82afbd0b', 'request_id': 'req-91528d4a-1701-4c75-b20c-ad0e2132048c'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:59:04.961274 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-91528d4a-1701-4c75-b20c-ad0e2132048c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '678aa241e39f4bea9fa9e73e82afbd0b', 'request_id': 'req-91528d4a-1701-4c75-b20c-ad0e2132048c'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:59:04.968127 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-91528d4a-1701-4c75-b20c-ad0e2132048c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '678aa241e39f4bea9fa9e73e82afbd0b', 'request_id': 'req-91528d4a-1701-4c75-b20c-ad0e2132048c'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:59:04.978440 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-91528d4a-1701-4c75-b20c-ad0e2132048c admin admin] The token cache is being invalidate because project 678aa241e39f4bea9fa9e73e82afbd0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 08:59:04.978732 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-91528d4a-1701-4c75-b20c-ad0e2132048c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:59:04.979374 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-91528d4a-1701-4c75-b20c-ad0e2132048c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:59:04.980887 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 971/1956] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 08:59:04 2022] DELETE /identity/v3/projects/678aa241e39f4bea9fa9e73e82afbd0b => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:59:04.984907 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a133e9-1081-4d6b-b1c8-7dfdeccc1fa3 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:59:04.986386 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-c3a133e9-1081-4d6b-b1c8-7dfdeccc1fa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:59:04.999906 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a133e9-1081-4d6b-b1c8-7dfdeccc1fa3 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:59:05.056096 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0569a685-a152-49d9-b4c0-7f646e874eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:59:05.056728 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0569a685-a152-49d9-b4c0-7f646e874eb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:59:05.057012 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0569a685-a152-49d9-b4c0-7f646e874eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:59:05.057290 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0569a685-a152-49d9-b4c0-7f646e874eb9 admin admin] PATH_INFO: `/v3/users/f6334153b91941b59ae8fa44e3073063` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:59:05.070236 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0569a685-a152-49d9-b4c0-7f646e874eb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6334153b91941b59ae8fa44e3073063)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:59:05.072148 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0569a685-a152-49d9-b4c0-7f646e874eb9 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:59:05.117204 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-0569a685-a152-49d9-b4c0-7f646e874eb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6334153b91941b59ae8fa44e3073063'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:59:05.117893 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-0569a685-a152-49d9-b4c0-7f646e874eb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6334153b91941b59ae8fa44e3073063'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:59:05.125521 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-0569a685-a152-49d9-b4c0-7f646e874eb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6334153b91941b59ae8fa44e3073063'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:59:05.138546 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-0569a685-a152-49d9-b4c0-7f646e874eb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6334153b91941b59ae8fa44e3073063'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:59:05.146086 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-0569a685-a152-49d9-b4c0-7f646e874eb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6334153b91941b59ae8fa44e3073063'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:59:05.151951 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 986/1957] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 08:59:04 2022] DELETE /identity/v3/users/f6334153b91941b59ae8fa44e3073063 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:59:05.388392 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91528d4a-1701-4c75-b20c-ad0e2132048c admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:59:05.390228 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-91528d4a-1701-4c75-b20c-ad0e2132048c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:59:05.406229 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91528d4a-1701-4c75-b20c-ad0e2132048c admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:59:05.464824 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c7a307-3716-4310-b6a5-ed7f110da303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:59:05.465454 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7a307-3716-4310-b6a5-ed7f110da303 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:59:05.465768 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7a307-3716-4310-b6a5-ed7f110da303 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:59:05.466048 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7a307-3716-4310-b6a5-ed7f110da303 admin admin] PATH_INFO: `/v3/projects/1df1d13607fb4c47970939aa8b1fd458` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:59:05.477395 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c7a307-3716-4310-b6a5-ed7f110da303 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1df1d13607fb4c47970939aa8b1fd458)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:59:05.479362 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c7a307-3716-4310-b6a5-ed7f110da303 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:59:05.487917 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:59:05.527758 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-92c7a307-3716-4310-b6a5-ed7f110da303 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1df1d13607fb4c47970939aa8b1fd458', 'request_id': 'req-92c7a307-3716-4310-b6a5-ed7f110da303'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:59:05.535065 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-92c7a307-3716-4310-b6a5-ed7f110da303 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1df1d13607fb4c47970939aa8b1fd458', 'request_id': 'req-92c7a307-3716-4310-b6a5-ed7f110da303'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:59:05.544409 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-92c7a307-3716-4310-b6a5-ed7f110da303 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1df1d13607fb4c47970939aa8b1fd458', 'request_id': 'req-92c7a307-3716-4310-b6a5-ed7f110da303'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:59:05.555558 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-92c7a307-3716-4310-b6a5-ed7f110da303 admin admin] The token cache is being invalidate because project 1df1d13607fb4c47970939aa8b1fd458 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 08:59:05.555850 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-92c7a307-3716-4310-b6a5-ed7f110da303 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:59:05.556497 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-92c7a307-3716-4310-b6a5-ed7f110da303 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 08:59:05.558040 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 972/1958] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 08:59:05 2022] DELETE /identity/v3/projects/1df1d13607fb4c47970939aa8b1fd458 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 08:59:15.148475 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=70,keystone:DELETE=13,keystone:INSERT=2 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:59:15.552484 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=21,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:59:33.186040 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0569a685-a152-49d9-b4c0-7f646e874eb9 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:59:33.187540 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-0569a685-a152-49d9-b4c0-7f646e874eb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:59:33.258834 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0569a685-a152-49d9-b4c0-7f646e874eb9 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:59:33.320041 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f10617-f08c-4b20-9963-e17301293607 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0a8af5cadc4cc1ab73e8eb2a20bb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:59:33.320652 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f10617-f08c-4b20-9963-e17301293607 service nova] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:59:33.320928 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f10617-f08c-4b20-9963-e17301293607 service nova] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:59:33.321201 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f10617-f08c-4b20-9963-e17301293607 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:59:33.322925 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-b4f10617-f08c-4b20-9963-e17301293607 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:59:33.378662 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f10617-f08c-4b20-9963-e17301293607 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:59:33.380237 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f10617-f08c-4b20-9963-e17301293607 service nova] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:59:33.389469 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 987/1959] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:59:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:59:41.290914 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c7a307-3716-4310-b6a5-ed7f110da303 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:59:41.292358 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-92c7a307-3716-4310-b6a5-ed7f110da303 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:59:41.347827 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c7a307-3716-4310-b6a5-ed7f110da303 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:59:41.351420 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcd4934-f883-4dd2-b170-d3eb6ed84db9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:59:41.352023 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd4934-f883-4dd2-b170-d3eb6ed84db9 service cinder] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:59:41.352305 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd4934-f883-4dd2-b170-d3eb6ed84db9 service cinder] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:59:41.352581 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd4934-f883-4dd2-b170-d3eb6ed84db9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:59:41.354307 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-6bcd4934-f883-4dd2-b170-d3eb6ed84db9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:59:41.413119 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcd4934-f883-4dd2-b170-d3eb6ed84db9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:59:41.414617 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcd4934-f883-4dd2-b170-d3eb6ed84db9 service cinder] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:59:41.423560 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 973/1960] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:59:41 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:59:43.385075 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=84 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 08:59:46.884029 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f10617-f08c-4b20-9963-e17301293607 service nova] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:59:46.885532 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-b4f10617-f08c-4b20-9963-e17301293607 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:59:46.940846 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f10617-f08c-4b20-9963-e17301293607 service nova] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:59:46.944353 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3d54eb-5a9f-4f68-a1d7-67e1d34b5adf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:59:46.944944 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d54eb-5a9f-4f68-a1d7-67e1d34b5adf service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:59:46.945223 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d54eb-5a9f-4f68-a1d7-67e1d34b5adf service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:59:46.945494 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d54eb-5a9f-4f68-a1d7-67e1d34b5adf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:59:46.947164 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-4b3d54eb-5a9f-4f68-a1d7-67e1d34b5adf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:59:47.003809 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3d54eb-5a9f-4f68-a1d7-67e1d34b5adf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:59:47.005759 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3d54eb-5a9f-4f68-a1d7-67e1d34b5adf service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:59:47.014545 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 988/1961] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:59:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:59:48.750524 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebf8096-61e3-4c7b-a69a-e89f494d449c None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:59:48.751064 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebf8096-61e3-4c7b-a69a-e89f494d449c None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:59:48.751168 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebf8096-61e3-4c7b-a69a-e89f494d449c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:59:48.779258 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-3ebf8096-61e3-4c7b-a69a-e89f494d449c None None] MFA Rules not processed for user `060d1fae9b0b488da1f7553cabd97d28`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 08:59:48.780473 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-3ebf8096-61e3-4c7b-a69a-e89f494d449c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:59:48.788260 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 974/1962] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 08:59:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 08:59:49.161823 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3d54eb-5a9f-4f68-a1d7-67e1d34b5adf service neutron] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:59:49.163238 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-4b3d54eb-5a9f-4f68-a1d7-67e1d34b5adf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:59:49.174092 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3d54eb-5a9f-4f68-a1d7-67e1d34b5adf service neutron] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:59:49.178190 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8591ff0-f5c2-4c5e-b94b-d75ce19a63c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:59:49.178785 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8591ff0-f5c2-4c5e-b94b-d75ce19a63c0 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:59:49.179065 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8591ff0-f5c2-4c5e-b94b-d75ce19a63c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:59:49.179339 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8591ff0-f5c2-4c5e-b94b-d75ce19a63c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:59:49.180999 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-d8591ff0-f5c2-4c5e-b94b-d75ce19a63c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:59:49.190074 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8591ff0-f5c2-4c5e-b94b-d75ce19a63c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:59:49.191476 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8591ff0-f5c2-4c5e-b94b-d75ce19a63c0 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:59:49.199815 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 989/1963] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:59:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:59:50.528968 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebf8096-61e3-4c7b-a69a-e89f494d449c None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:59:50.530368 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-3ebf8096-61e3-4c7b-a69a-e89f494d449c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:59:50.601108 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebf8096-61e3-4c7b-a69a-e89f494d449c None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:59:50.608562 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6277f40-4041-4b7e-a17c-466d4e5eeeeb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38986ca2612c4fc59d960cd693834f9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:59:50.609169 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6277f40-4041-4b7e-a17c-466d4e5eeeeb service placement] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:59:50.609449 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6277f40-4041-4b7e-a17c-466d4e5eeeeb service placement] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:59:50.609744 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6277f40-4041-4b7e-a17c-466d4e5eeeeb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:59:50.611473 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-e6277f40-4041-4b7e-a17c-466d4e5eeeeb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:59:50.623036 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6277f40-4041-4b7e-a17c-466d4e5eeeeb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:59:50.624640 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6277f40-4041-4b7e-a17c-466d4e5eeeeb service placement] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:59:50.634230 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 975/1964] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:59:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:59:54.150704 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8591ff0-f5c2-4c5e-b94b-d75ce19a63c0 service neutron] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:59:54.152132 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-d8591ff0-f5c2-4c5e-b94b-d75ce19a63c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:59:54.164053 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8591ff0-f5c2-4c5e-b94b-d75ce19a63c0 service neutron] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:59:54.167566 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a61d76-f1ce-4abd-8649-0ffe0364d1d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:59:54.168160 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a61d76-f1ce-4abd-8649-0ffe0364d1d4 service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:59:54.168443 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a61d76-f1ce-4abd-8649-0ffe0364d1d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:59:54.168718 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a61d76-f1ce-4abd-8649-0ffe0364d1d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:59:54.170662 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-13a61d76-f1ce-4abd-8649-0ffe0364d1d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:59:54.234641 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a61d76-f1ce-4abd-8649-0ffe0364d1d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:59:54.236103 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a61d76-f1ce-4abd-8649-0ffe0364d1d4 service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:59:54.245353 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 990/1965] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:59:54 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 08:59:59.337249 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6277f40-4041-4b7e-a17c-466d4e5eeeeb service placement] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 08:59:59.338702 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-e6277f40-4041-4b7e-a17c-466d4e5eeeeb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:59:59.353592 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6277f40-4041-4b7e-a17c-466d4e5eeeeb service placement] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 08:59:59.359032 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a7bd37-a955-4277-9433-dc9553a3ee28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0a8af5cadc4cc1ab73e8eb2a20bb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 08:59:59.359636 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a7bd37-a955-4277-9433-dc9553a3ee28 service nova] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 08:59:59.359918 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a7bd37-a955-4277-9433-dc9553a3ee28 service nova] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 08:59:59.360191 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a7bd37-a955-4277-9433-dc9553a3ee28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 08:59:59.361939 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-47a7bd37-a955-4277-9433-dc9553a3ee28 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 08:59:59.374383 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a7bd37-a955-4277-9433-dc9553a3ee28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 08:59:59.375980 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a7bd37-a955-4277-9433-dc9553a3ee28 service nova] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 08:59:59.391914 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 976/1966] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 08:59:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:00:04.241099 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=82 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:00:09.382615 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=92 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:00:52.826382 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a61d76-f1ce-4abd-8649-0ffe0364d1d4 service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:00:52.835487 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a61d76-f1ce-4abd-8649-0ffe0364d1d4 service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:00:52.839700 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7265ef-f132-443e-bc57-157d8b558585 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:00:52.840303 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7265ef-f132-443e-bc57-157d8b558585 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:00:52.840589 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7265ef-f132-443e-bc57-157d8b558585 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:00:52.840854 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7265ef-f132-443e-bc57-157d8b558585 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:00:52.842609 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-6a7265ef-f132-443e-bc57-157d8b558585 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:00:52.852015 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7265ef-f132-443e-bc57-157d8b558585 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:00:52.853432 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7265ef-f132-443e-bc57-157d8b558585 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:00:52.861660 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 991/1967] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:00:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:01:02.195791 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a7bd37-a955-4277-9433-dc9553a3ee28 service nova] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:01:02.207388 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a7bd37-a955-4277-9433-dc9553a3ee28 service nova] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:01:02.212722 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4059db61-4e48-4e82-8f81-2e30ccaf850e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0a8af5cadc4cc1ab73e8eb2a20bb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:01:02.213313 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4059db61-4e48-4e82-8f81-2e30ccaf850e service nova] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:01:02.213590 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4059db61-4e48-4e82-8f81-2e30ccaf850e service nova] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:01:02.213899 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4059db61-4e48-4e82-8f81-2e30ccaf850e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:01:02.215574 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-4059db61-4e48-4e82-8f81-2e30ccaf850e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:01:02.274796 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4059db61-4e48-4e82-8f81-2e30ccaf850e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:01:02.276381 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4059db61-4e48-4e82-8f81-2e30ccaf850e service nova] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:01:02.286222 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 977/1968] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:01:02 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:01:02.858060 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=6 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:01:12.281468 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=28 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:03:39.852749 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7265ef-f132-443e-bc57-157d8b558585 service neutron] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:03:39.871908 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7265ef-f132-443e-bc57-157d8b558585 service neutron] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:03:39.876212 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d83f7f-6842-42c3-8746-89834cfb05f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:03:39.876821 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d83f7f-6842-42c3-8746-89834cfb05f8 service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:03:39.877107 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d83f7f-6842-42c3-8746-89834cfb05f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:03:39.877388 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d83f7f-6842-42c3-8746-89834cfb05f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:03:39.879158 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-60d83f7f-6842-42c3-8746-89834cfb05f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:03:39.964759 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d83f7f-6842-42c3-8746-89834cfb05f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:03:39.966221 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d83f7f-6842-42c3-8746-89834cfb05f8 service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:03:39.974999 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 992/1969] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:03:39 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:03:48.982239 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4059db61-4e48-4e82-8f81-2e30ccaf850e service nova] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:03:48.983689 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-4059db61-4e48-4e82-8f81-2e30ccaf850e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:03:49.072749 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4059db61-4e48-4e82-8f81-2e30ccaf850e service nova] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:03:49.077953 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f09fa2f-9a3b-4f09-9b8f-93fb3d4fda94 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38986ca2612c4fc59d960cd693834f9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:03:49.079150 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f09fa2f-9a3b-4f09-9b8f-93fb3d4fda94 service placement] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:03:49.079432 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f09fa2f-9a3b-4f09-9b8f-93fb3d4fda94 service placement] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:03:49.079706 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f09fa2f-9a3b-4f09-9b8f-93fb3d4fda94 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:03:49.081620 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-0f09fa2f-9a3b-4f09-9b8f-93fb3d4fda94 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:03:49.095235 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f09fa2f-9a3b-4f09-9b8f-93fb3d4fda94 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:03:49.096839 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f09fa2f-9a3b-4f09-9b8f-93fb3d4fda94 service placement] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:03:49.107040 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 978/1970] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:03:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:03:49.970938 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=43 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:03:54.904644 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d83f7f-6842-42c3-8746-89834cfb05f8 service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:03:54.924550 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d83f7f-6842-42c3-8746-89834cfb05f8 service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:03:54.928180 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882166b2-a883-4166-a643-f7a116549508 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:03:54.928783 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882166b2-a883-4166-a643-f7a116549508 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:03:54.929066 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882166b2-a883-4166-a643-f7a116549508 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:03:54.929341 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882166b2-a883-4166-a643-f7a116549508 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:03:54.931092 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-882166b2-a883-4166-a643-f7a116549508 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:03:54.964941 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882166b2-a883-4166-a643-f7a116549508 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:03:54.966333 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882166b2-a883-4166-a643-f7a116549508 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:03:54.986917 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 993/1971] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:03:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:03:56.233644 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f09fa2f-9a3b-4f09-9b8f-93fb3d4fda94 service placement] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:03:56.243506 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f09fa2f-9a3b-4f09-9b8f-93fb3d4fda94 service placement] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:03:56.247048 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c651fd6-4afb-4123-b562-ec98e8259e78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:03:56.247638 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c651fd6-4afb-4123-b562-ec98e8259e78 service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:03:56.247913 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c651fd6-4afb-4123-b562-ec98e8259e78 service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:03:56.248185 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c651fd6-4afb-4123-b562-ec98e8259e78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:03:56.249879 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-7c651fd6-4afb-4123-b562-ec98e8259e78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:03:56.262577 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c651fd6-4afb-4123-b562-ec98e8259e78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:03:56.263989 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c651fd6-4afb-4123-b562-ec98e8259e78 service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:03:56.275017 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 979/1972] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:03:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:04:04.984116 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=22 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:04:06.270310 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=43 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:04:33.736678 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882166b2-a883-4166-a643-f7a116549508 service neutron] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:04:33.746280 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882166b2-a883-4166-a643-f7a116549508 service neutron] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:04:33.749955 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd324a4-1d69-4455-bfa5-081c106307fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:04:33.750553 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd324a4-1d69-4455-bfa5-081c106307fe service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:04:33.750837 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd324a4-1d69-4455-bfa5-081c106307fe service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:04:33.751113 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd324a4-1d69-4455-bfa5-081c106307fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:04:33.759267 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd324a4-1d69-4455-bfa5-081c106307fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:04:33.760669 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd324a4-1d69-4455-bfa5-081c106307fe service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:04:33.769694 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 994/1973] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:04:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:04:42.334016 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c651fd6-4afb-4123-b562-ec98e8259e78 service neutron] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:04:42.352767 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c651fd6-4afb-4123-b562-ec98e8259e78 service neutron] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:04:42.356517 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1077df15-b87d-48bd-97a6-128d04e5b291 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:04:42.357118 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1077df15-b87d-48bd-97a6-128d04e5b291 service cinder] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:04:42.357398 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1077df15-b87d-48bd-97a6-128d04e5b291 service cinder] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:04:42.357672 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1077df15-b87d-48bd-97a6-128d04e5b291 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:04:42.366619 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1077df15-b87d-48bd-97a6-128d04e5b291 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:04:42.368053 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1077df15-b87d-48bd-97a6-128d04e5b291 service cinder] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:04:42.395327 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 980/1974] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:04:42 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:04:43.765287 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=6 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:04:46.506056 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd324a4-1d69-4455-bfa5-081c106307fe service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:04:46.524387 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd324a4-1d69-4455-bfa5-081c106307fe service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:04:46.528010 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866ae5fc-963b-4389-b090-99d0b9525af2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:04:46.528605 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ae5fc-963b-4389-b090-99d0b9525af2 service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:04:46.528891 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ae5fc-963b-4389-b090-99d0b9525af2 service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:04:46.529164 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ae5fc-963b-4389-b090-99d0b9525af2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:04:46.538187 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866ae5fc-963b-4389-b090-99d0b9525af2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:04:46.539595 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866ae5fc-963b-4389-b090-99d0b9525af2 service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:04:46.566389 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 995/1975] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:04:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:04:52.392720 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=17 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:04:55.067045 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1077df15-b87d-48bd-97a6-128d04e5b291 service cinder] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:04:55.077646 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1077df15-b87d-48bd-97a6-128d04e5b291 service cinder] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:04:55.082486 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883975de-afe0-4168-a6cc-fa561435ee76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:04:55.083340 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883975de-afe0-4168-a6cc-fa561435ee76 service cinder] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:04:55.083871 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883975de-afe0-4168-a6cc-fa561435ee76 service cinder] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:04:55.084238 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883975de-afe0-4168-a6cc-fa561435ee76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:04:55.095114 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883975de-afe0-4168-a6cc-fa561435ee76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:04:55.096585 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883975de-afe0-4168-a6cc-fa561435ee76 service cinder] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:04:55.125490 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 981/1976] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:04:55 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:04:56.563793 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=17 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:05:05.122659 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=13 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:05:52.204404 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866ae5fc-963b-4389-b090-99d0b9525af2 service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:05:52.208169 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-866ae5fc-963b-4389-b090-99d0b9525af2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:05:52.231184 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866ae5fc-963b-4389-b090-99d0b9525af2 service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:05:52.244690 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f809c48c-e9aa-4fbb-8827-feeb1e987197 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0a8af5cadc4cc1ab73e8eb2a20bb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:05:52.250473 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809c48c-e9aa-4fbb-8827-feeb1e987197 service nova] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:05:52.251273 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809c48c-e9aa-4fbb-8827-feeb1e987197 service nova] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:05:52.251715 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809c48c-e9aa-4fbb-8827-feeb1e987197 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:05:52.254532 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-f809c48c-e9aa-4fbb-8827-feeb1e987197 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:05:52.272602 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f809c48c-e9aa-4fbb-8827-feeb1e987197 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:05:52.274246 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f809c48c-e9aa-4fbb-8827-feeb1e987197 service nova] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:05:52.286771 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 996/1977] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:05:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:05:56.020139 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883975de-afe0-4168-a6cc-fa561435ee76 service cinder] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:05:56.029485 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883975de-afe0-4168-a6cc-fa561435ee76 service cinder] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:05:56.033652 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc8b4f7-8b84-46f2-bfac-60a888522f9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:05:56.034276 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc8b4f7-8b84-46f2-bfac-60a888522f9b service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:05:56.034554 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc8b4f7-8b84-46f2-bfac-60a888522f9b service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:05:56.034826 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc8b4f7-8b84-46f2-bfac-60a888522f9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:05:56.036769 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-fbc8b4f7-8b84-46f2-bfac-60a888522f9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:05:56.119287 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc8b4f7-8b84-46f2-bfac-60a888522f9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:05:56.120711 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc8b4f7-8b84-46f2-bfac-60a888522f9b service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:05:56.138976 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 982/1978] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:05:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:05:56.697176 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f809c48c-e9aa-4fbb-8827-feeb1e987197 service nova] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:05:56.706593 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f809c48c-e9aa-4fbb-8827-feeb1e987197 service nova] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:05:56.710245 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fb886c-b594-4a95-86e8-f0914bd0e285 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:05:56.710843 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb886c-b594-4a95-86e8-f0914bd0e285 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:05:56.711127 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb886c-b594-4a95-86e8-f0914bd0e285 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:05:56.711408 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb886c-b594-4a95-86e8-f0914bd0e285 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:05:56.719101 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fb886c-b594-4a95-86e8-f0914bd0e285 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:05:56.720620 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fb886c-b594-4a95-86e8-f0914bd0e285 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:05:56.728923 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 997/1979] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:05:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 09:05:59.695183 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf68a5-0a9f-4efa-9d30-ada16fbf3429 None None] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:05:59.696314 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf68a5-0a9f-4efa-9d30-ada16fbf3429 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:05:59.696761 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf68a5-0a9f-4efa-9d30-ada16fbf3429 None None] PATH_INFO: `/` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:05:59.698865 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 983/1980] 10.222.0.30 () {54 vars in 921 bytes} [Sat Oct 15 09:05:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 09:05:59.746502 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8577e66e-4c59-48fc-b464-6a4db8b24d6b None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:05:59.747410 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8577e66e-4c59-48fc-b464-6a4db8b24d6b None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:05:59.747954 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8577e66e-4c59-48fc-b464-6a4db8b24d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:05:59.795389 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bf68a5-0a9f-4efa-9d30-ada16fbf3429 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:05:59.800296 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-c2bf68a5-0a9f-4efa-9d30-ada16fbf3429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:05:59.816278 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-8577e66e-4c59-48fc-b464-6a4db8b24d6b None None] MFA Rules not processed for user `38986ca2612c4fc59d960cd693834f9b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:05:59.817613 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-8577e66e-4c59-48fc-b464-6a4db8b24d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:05:59.818047 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bf68a5-0a9f-4efa-9d30-ada16fbf3429 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:05:59.822583 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcb31af-8145-4369-992e-0d92902f7004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:05:59.823233 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb31af-8145-4369-992e-0d92902f7004 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:05:59.823586 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb31af-8145-4369-992e-0d92902f7004 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:05:59.823957 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb31af-8145-4369-992e-0d92902f7004 admin admin] PATH_INFO: `/v3/users/9cc9e2cf11804cc39a7de3c314e46679` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:05:59.826223 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 998/1981] 10.222.0.30 () {58 vars in 1018 bytes} [Sat Oct 15 09:05:59 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:05:59.837383 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8577e66e-4c59-48fc-b464-6a4db8b24d6b None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:05:59.838933 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcb31af-8145-4369-992e-0d92902f7004 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cc9e2cf11804cc39a7de3c314e46679)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:05:59.840941 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcb31af-8145-4369-992e-0d92902f7004 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:05:59.847669 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8577e66e-4c59-48fc-b464-6a4db8b24d6b None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:05:59.852730 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b248de-4fcf-4fcd-a8b6-a28a6dfa3991 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38986ca2612c4fc59d960cd693834f9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:05:59.853348 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b248de-4fcf-4fcd-a8b6-a28a6dfa3991 service placement] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:05:59.853636 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b248de-4fcf-4fcd-a8b6-a28a6dfa3991 service placement] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:05:59.853942 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b248de-4fcf-4fcd-a8b6-a28a6dfa3991 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:05:59.862911 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b248de-4fcf-4fcd-a8b6-a28a6dfa3991 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:05:59.864472 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b248de-4fcf-4fcd-a8b6-a28a6dfa3991 service placement] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:05:59.876543 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 999/1982] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:05:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 09:05:59.889981 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-abcb31af-8145-4369-992e-0d92902f7004 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cc9e2cf11804cc39a7de3c314e46679'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:05:59.890837 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-abcb31af-8145-4369-992e-0d92902f7004 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cc9e2cf11804cc39a7de3c314e46679'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:05:59.903166 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-abcb31af-8145-4369-992e-0d92902f7004 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cc9e2cf11804cc39a7de3c314e46679'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:05:59.911006 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-abcb31af-8145-4369-992e-0d92902f7004 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cc9e2cf11804cc39a7de3c314e46679'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:05:59.911639 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-abcb31af-8145-4369-992e-0d92902f7004 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cc9e2cf11804cc39a7de3c314e46679'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:05:59.924535 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 984/1983] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 09:05:59 2022] DELETE /identity/v3/users/9cc9e2cf11804cc39a7de3c314e46679 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:06:00.093893 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b248de-4fcf-4fcd-a8b6-a28a6dfa3991 service placement] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:06:00.109462 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b248de-4fcf-4fcd-a8b6-a28a6dfa3991 service placement] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:06:00.166333 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad54ab2-1e4a-4388-a147-3095ea3d6d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:06:00.166965 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad54ab2-1e4a-4388-a147-3095ea3d6d53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:06:00.167253 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad54ab2-1e4a-4388-a147-3095ea3d6d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:06:00.167533 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad54ab2-1e4a-4388-a147-3095ea3d6d53 admin admin] PATH_INFO: `/v3/projects/f67051bce2414ff791a2a1e1a638cadc` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:06:00.168650 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad54ab2-1e4a-4388-a147-3095ea3d6d53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f67051bce2414ff791a2a1e1a638cadc)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:06:00.170584 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad54ab2-1e4a-4388-a147-3095ea3d6d53 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:06:00.215377 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-9ad54ab2-1e4a-4388-a147-3095ea3d6d53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f67051bce2414ff791a2a1e1a638cadc', 'request_id': 'req-9ad54ab2-1e4a-4388-a147-3095ea3d6d53'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:06:00.221566 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-9ad54ab2-1e4a-4388-a147-3095ea3d6d53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f67051bce2414ff791a2a1e1a638cadc', 'request_id': 'req-9ad54ab2-1e4a-4388-a147-3095ea3d6d53'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:06:00.228444 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-9ad54ab2-1e4a-4388-a147-3095ea3d6d53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f67051bce2414ff791a2a1e1a638cadc', 'request_id': 'req-9ad54ab2-1e4a-4388-a147-3095ea3d6d53'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:06:00.238619 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-9ad54ab2-1e4a-4388-a147-3095ea3d6d53 admin admin] The token cache is being invalidate because project f67051bce2414ff791a2a1e1a638cadc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 09:06:00.239041 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-9ad54ab2-1e4a-4388-a147-3095ea3d6d53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:06:00.239677 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-9ad54ab2-1e4a-4388-a147-3095ea3d6d53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:06:00.241202 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1000/1984] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 09:06:00 2022] DELETE /identity/v3/projects/f67051bce2414ff791a2a1e1a638cadc => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:06:00.246872 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c9f884-17fe-47c5-a229-c6bfc9d6c72c None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:06:00.247160 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c9f884-17fe-47c5-a229-c6bfc9d6c72c None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:06:00.247436 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c9f884-17fe-47c5-a229-c6bfc9d6c72c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:06:00.287844 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:06:00.296137 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-53c9f884-17fe-47c5-a229-c6bfc9d6c72c None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:06:00.298516 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-53c9f884-17fe-47c5-a229-c6bfc9d6c72c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:06:00.367378 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 985/1985] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:06:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:06:00.368881 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd442882-d46a-466f-adf8-31af7d63a7c4 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:06:00.369174 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd442882-d46a-466f-adf8-31af7d63a7c4 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:06:00.369454 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd442882-d46a-466f-adf8-31af7d63a7c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:06:00.388985 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-bd442882-d46a-466f-adf8-31af7d63a7c4 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:06:00.390224 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-bd442882-d46a-466f-adf8-31af7d63a7c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:06:00.398623 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1001/1986] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:06:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:06:00.402834 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c9f884-17fe-47c5-a229-c6bfc9d6c72c None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:06:00.413427 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c9f884-17fe-47c5-a229-c6bfc9d6c72c None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:06:00.417982 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb4602-f9a4-4fa5-b459-8da08b391917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:06:00.418583 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb4602-f9a4-4fa5-b459-8da08b391917 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:06:00.418867 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb4602-f9a4-4fa5-b459-8da08b391917 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:06:00.419145 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb4602-f9a4-4fa5-b459-8da08b391917 admin admin] PATH_INFO: `/v3/domains` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:06:00.419822 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eb4602-f9a4-4fa5-b459-8da08b391917 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 09:06:00.420132 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eb4602-f9a4-4fa5-b459-8da08b391917 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:06:00.421776 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eb4602-f9a4-4fa5-b459-8da08b391917 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:06:00.433371 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 986/1987] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 09:06:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 09:06:00.437780 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd442882-d46a-466f-adf8-31af7d63a7c4 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:06:00.446977 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd442882-d46a-466f-adf8-31af7d63a7c4 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:06:00.451541 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6f8f3b-6871-4671-8e27-b4ef84876b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:06:00.452140 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6f8f3b-6871-4671-8e27-b4ef84876b4d admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:06:00.452420 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6f8f3b-6871-4671-8e27-b4ef84876b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:06:00.452694 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6f8f3b-6871-4671-8e27-b4ef84876b4d admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:06:00.453886 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6f8f3b-6871-4671-8e27-b4ef84876b4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:06:00.455781 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6f8f3b-6871-4671-8e27-b4ef84876b4d admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:06:00.460248 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:06:00.487483 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1002/1988] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 09:06:00 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:06:00.489375 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb4602-f9a4-4fa5-b459-8da08b391917 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:06:00.499315 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb4602-f9a4-4fa5-b459-8da08b391917 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:06:00.557929 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5200c362-a9dd-4c9a-aa1d-e90d3f4e333f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:06:00.558605 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5200c362-a9dd-4c9a-aa1d-e90d3f4e333f admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:06:00.558889 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5200c362-a9dd-4c9a-aa1d-e90d3f4e333f admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:06:00.559168 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5200c362-a9dd-4c9a-aa1d-e90d3f4e333f admin admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:06:00.559934 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5200c362-a9dd-4c9a-aa1d-e90d3f4e333f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:06:00.561876 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5200c362-a9dd-4c9a-aa1d-e90d3f4e333f admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:06:00.586750 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 987/1989] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 09:06:00 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:06:00.587827 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6f8f3b-6871-4671-8e27-b4ef84876b4d admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:06:00.598676 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6f8f3b-6871-4671-8e27-b4ef84876b4d admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:06:00.603419 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c78153d-43b9-4435-8e6c-58fed05f6c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:06:00.604020 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c78153d-43b9-4435-8e6c-58fed05f6c65 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:06:00.604304 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c78153d-43b9-4435-8e6c-58fed05f6c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:06:00.604580 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c78153d-43b9-4435-8e6c-58fed05f6c65 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:06:00.605222 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c78153d-43b9-4435-8e6c-58fed05f6c65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:06:00.607156 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c78153d-43b9-4435-8e6c-58fed05f6c65 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:06:00.620872 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1003/1990] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:06:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:06:00.622191 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5200c362-a9dd-4c9a-aa1d-e90d3f4e333f admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:06:00.632718 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5200c362-a9dd-4c9a-aa1d-e90d3f4e333f admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:06:00.638198 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfb1ac0-f3af-4c97-b685-5b8041c1b4c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:06:00.638812 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb1ac0-f3af-4c97-b685-5b8041c1b4c2 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:06:00.639101 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb1ac0-f3af-4c97-b685-5b8041c1b4c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:06:00.639380 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb1ac0-f3af-4c97-b685-5b8041c1b4c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:06:00.640030 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfb1ac0-f3af-4c97-b685-5b8041c1b4c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:06:00.641657 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfb1ac0-f3af-4c97-b685-5b8041c1b4c2 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:06:00.653716 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 988/1991] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:06:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:06:00.657752 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c78153d-43b9-4435-8e6c-58fed05f6c65 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:06:00.668034 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c78153d-43b9-4435-8e6c-58fed05f6c65 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:06:00.673248 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e4d3a-af6a-4053-984a-6ab73f2eb6a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:06:00.673933 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e4d3a-af6a-4053-984a-6ab73f2eb6a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:06:00.674215 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e4d3a-af6a-4053-984a-6ab73f2eb6a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:06:00.674494 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e4d3a-af6a-4053-984a-6ab73f2eb6a2 admin admin] PATH_INFO: `/v3/projects/0ade5da44ad948fdb77558898be6bd38/users/525f7c26e8e74e17bf07d5d62150ad56/roles/3248f0268738489b883041851d681bb7` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:06:00.690609 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9e4d3a-af6a-4053-984a-6ab73f2eb6a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ade5da44ad948fdb77558898be6bd38, user_id=525f7c26e8e74e17bf07d5d62150ad56, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:06:00.693264 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9e4d3a-af6a-4053-984a-6ab73f2eb6a2 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:06:00.706979 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1004/1992] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:06:00 2022] PUT /identity/v3/projects/0ade5da44ad948fdb77558898be6bd38/users/525f7c26e8e74e17bf07d5d62150ad56/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 09:06:00.717623 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403dad9b-57c5-4bc2-996a-5593c48969a2 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:06:00.718128 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403dad9b-57c5-4bc2-996a-5593c48969a2 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:06:00.718250 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403dad9b-57c5-4bc2-996a-5593c48969a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:06:00.750499 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-403dad9b-57c5-4bc2-996a-5593c48969a2 None None] MFA Rules not processed for user `525f7c26e8e74e17bf07d5d62150ad56`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:06:00.751708 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-403dad9b-57c5-4bc2-996a-5593c48969a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:06:00.781618 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e4d3a-af6a-4053-984a-6ab73f2eb6a2 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:06:00.784024 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-cc9e4d3a-af6a-4053-984a-6ab73f2eb6a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:06:00.813617 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:06:00.818123 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 989/1993] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:06:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:06:00.829064 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403dad9b-57c5-4bc2-996a-5593c48969a2 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:06:00.830489 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-403dad9b-57c5-4bc2-996a-5593c48969a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:06:00.842468 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e4d3a-af6a-4053-984a-6ab73f2eb6a2 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:06:00.846678 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996c8640-b7ed-4708-80c2-6baceda12501 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:06:00.847305 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c8640-b7ed-4708-80c2-6baceda12501 service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:06:00.847567 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c8640-b7ed-4708-80c2-6baceda12501 service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:06:00.847845 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c8640-b7ed-4708-80c2-6baceda12501 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:06:00.849577 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-996c8640-b7ed-4708-80c2-6baceda12501 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:06:00.907337 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403dad9b-57c5-4bc2-996a-5593c48969a2 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:06:00.910168 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996c8640-b7ed-4708-80c2-6baceda12501 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:06:00.911894 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996c8640-b7ed-4708-80c2-6baceda12501 service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:06:00.913048 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4439d8-ea5c-4466-8907-bde57bb5ce37 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0a8af5cadc4cc1ab73e8eb2a20bb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:06:00.913659 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4439d8-ea5c-4466-8907-bde57bb5ce37 service nova] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:06:00.913969 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4439d8-ea5c-4466-8907-bde57bb5ce37 service nova] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:06:00.914242 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4439d8-ea5c-4466-8907-bde57bb5ce37 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:06:00.920948 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1005/1994] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:06:00 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 09:06:00.923272 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4439d8-ea5c-4466-8907-bde57bb5ce37 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:06:00.925405 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4439d8-ea5c-4466-8907-bde57bb5ce37 service nova] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:06:00.934797 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 990/1995] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:06:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 09:06:10.917209 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=69,keystone:INSERT=2 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:06:10.930967 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=34 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:08:52.936137 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996c8640-b7ed-4708-80c2-6baceda12501 service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:08:52.937543 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-996c8640-b7ed-4708-80c2-6baceda12501 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:08:53.006244 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996c8640-b7ed-4708-80c2-6baceda12501 service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:08:53.011331 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19378b10-935a-468e-9fcc-fc868caf55f7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38986ca2612c4fc59d960cd693834f9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:08:53.011938 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19378b10-935a-468e-9fcc-fc868caf55f7 service placement] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:08:53.012217 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19378b10-935a-468e-9fcc-fc868caf55f7 service placement] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:08:53.012490 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19378b10-935a-468e-9fcc-fc868caf55f7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:08:53.014207 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-19378b10-935a-468e-9fcc-fc868caf55f7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:08:53.025799 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19378b10-935a-468e-9fcc-fc868caf55f7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:08:53.027391 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19378b10-935a-468e-9fcc-fc868caf55f7 service placement] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:08:53.037867 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1006/1996] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:08:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:08:55.397393 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4439d8-ea5c-4466-8907-bde57bb5ce37 service nova] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:08:55.398901 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-ea4439d8-ea5c-4466-8907-bde57bb5ce37 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:08:55.466567 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4439d8-ea5c-4466-8907-bde57bb5ce37 service nova] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:08:55.470848 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7847fda1-4198-4022-a06a-6886acb90d84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:08:55.471451 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847fda1-4198-4022-a06a-6886acb90d84 service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:08:55.471735 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847fda1-4198-4022-a06a-6886acb90d84 service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:08:55.472009 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847fda1-4198-4022-a06a-6886acb90d84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:08:55.473733 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-7847fda1-4198-4022-a06a-6886acb90d84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:08:55.487278 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7847fda1-4198-4022-a06a-6886acb90d84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:08:55.488699 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7847fda1-4198-4022-a06a-6886acb90d84 service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:08:55.499872 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 991/1997] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:08:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:09:03.033486 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=32 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:09:05.494570 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=31 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:09:46.089658 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19378b10-935a-468e-9fcc-fc868caf55f7 service placement] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:09:46.091076 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-19378b10-935a-468e-9fcc-fc868caf55f7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:09:46.114764 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19378b10-935a-468e-9fcc-fc868caf55f7 service placement] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:09:46.118251 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeaa7d5-dd18-42e4-97aa-c0ab6f71d81b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:09:46.118838 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeaa7d5-dd18-42e4-97aa-c0ab6f71d81b service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:09:46.119112 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeaa7d5-dd18-42e4-97aa-c0ab6f71d81b service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:09:46.119378 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeaa7d5-dd18-42e4-97aa-c0ab6f71d81b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:09:46.121008 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-feeaa7d5-dd18-42e4-97aa-c0ab6f71d81b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:09:46.188341 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeaa7d5-dd18-42e4-97aa-c0ab6f71d81b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:09:46.189783 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeaa7d5-dd18-42e4-97aa-c0ab6f71d81b service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:09:46.198710 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1007/1998] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:09:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:09:46.240258 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7847fda1-4198-4022-a06a-6886acb90d84 service neutron] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:09:46.249367 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7847fda1-4198-4022-a06a-6886acb90d84 service neutron] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:09:46.252867 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a3390c-7f32-408e-888e-bc41a8f57c26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:09:46.253485 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a3390c-7f32-408e-888e-bc41a8f57c26 service cinder] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:09:46.253801 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a3390c-7f32-408e-888e-bc41a8f57c26 service cinder] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:09:46.254078 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a3390c-7f32-408e-888e-bc41a8f57c26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:09:46.255761 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-c8a3390c-7f32-408e-888e-bc41a8f57c26 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:09:46.269068 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a3390c-7f32-408e-888e-bc41a8f57c26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:09:46.270529 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a3390c-7f32-408e-888e-bc41a8f57c26 service cinder] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:09:46.281694 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 992/1999] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:09:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:09:46.323783 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeaa7d5-dd18-42e4-97aa-c0ab6f71d81b service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:09:46.328189 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a3390c-7f32-408e-888e-bc41a8f57c26 service cinder] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:09:46.329541 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-c8a3390c-7f32-408e-888e-bc41a8f57c26 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:09:46.333136 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeaa7d5-dd18-42e4-97aa-c0ab6f71d81b service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:09:46.336822 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca07d53-eee4-4afa-b928-06acf62e9097 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:09:46.337547 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca07d53-eee4-4afa-b928-06acf62e9097 service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:09:46.337742 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca07d53-eee4-4afa-b928-06acf62e9097 service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:09:46.338026 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca07d53-eee4-4afa-b928-06acf62e9097 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:09:46.339784 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-5ca07d53-eee4-4afa-b928-06acf62e9097 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:09:46.342978 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a3390c-7f32-408e-888e-bc41a8f57c26 service cinder] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:09:46.348418 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f44d90-94e4-4dba-959a-010d36bd955d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0a8af5cadc4cc1ab73e8eb2a20bb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:09:46.349010 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f44d90-94e4-4dba-959a-010d36bd955d service nova] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:09:46.349287 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f44d90-94e4-4dba-959a-010d36bd955d service nova] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:09:46.349553 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f44d90-94e4-4dba-959a-010d36bd955d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:09:46.351273 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-d5f44d90-94e4-4dba-959a-010d36bd955d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:09:46.428902 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca07d53-eee4-4afa-b928-06acf62e9097 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:09:46.431731 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca07d53-eee4-4afa-b928-06acf62e9097 service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:09:46.432414 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f44d90-94e4-4dba-959a-010d36bd955d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:09:46.434040 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f44d90-94e4-4dba-959a-010d36bd955d service nova] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:09:46.450815 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1008/2000] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:09:46 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:09:46.454099 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 993/2001] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:09:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 09:09:56.448098 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=79 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:09:56.450329 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=47 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:10:33.072940 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca07d53-eee4-4afa-b928-06acf62e9097 service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:10:33.082386 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca07d53-eee4-4afa-b928-06acf62e9097 service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:10:33.087405 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5cb2ef-51f9-4cdf-9ca6-e8209d7d8e9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:10:33.088024 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5cb2ef-51f9-4cdf-9ca6-e8209d7d8e9e service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:10:33.088507 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5cb2ef-51f9-4cdf-9ca6-e8209d7d8e9e service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:10:33.088799 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5cb2ef-51f9-4cdf-9ca6-e8209d7d8e9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:10:33.090640 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-5d5cb2ef-51f9-4cdf-9ca6-e8209d7d8e9e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:10:33.164094 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5cb2ef-51f9-4cdf-9ca6-e8209d7d8e9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:10:33.165515 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5cb2ef-51f9-4cdf-9ca6-e8209d7d8e9e service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:10:33.175454 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1009/2002] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:10:33 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:10:43.171194 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=33 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:11:34.357767 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f44d90-94e4-4dba-959a-010d36bd955d service nova] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:11:34.369320 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f44d90-94e4-4dba-959a-010d36bd955d service nova] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:11:34.375341 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4face12f-1584-46ba-acba-6537112b3416 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0a8af5cadc4cc1ab73e8eb2a20bb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:11:34.375945 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4face12f-1584-46ba-acba-6537112b3416 service nova] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:11:34.376227 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4face12f-1584-46ba-acba-6537112b3416 service nova] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:11:34.376502 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4face12f-1584-46ba-acba-6537112b3416 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:11:34.384267 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4face12f-1584-46ba-acba-6537112b3416 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:11:34.385891 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4face12f-1584-46ba-acba-6537112b3416 service nova] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:11:34.396383 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 994/2003] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:11:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:11:44.391818 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=6 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:11:46.847279 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5cb2ef-51f9-4cdf-9ca6-e8209d7d8e9e service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:11:46.855893 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5cb2ef-51f9-4cdf-9ca6-e8209d7d8e9e service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:11:46.860201 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64b7334-35f3-47cc-bd0c-d110b718fc1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:11:46.860872 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64b7334-35f3-47cc-bd0c-d110b718fc1a service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:11:46.861219 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64b7334-35f3-47cc-bd0c-d110b718fc1a service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:11:46.861501 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64b7334-35f3-47cc-bd0c-d110b718fc1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:11:46.870174 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64b7334-35f3-47cc-bd0c-d110b718fc1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:11:46.871619 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64b7334-35f3-47cc-bd0c-d110b718fc1a service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:11:46.900371 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1010/2004] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:11:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:11:47.277420 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4face12f-1584-46ba-acba-6537112b3416 service nova] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:11:47.279266 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-4face12f-1584-46ba-acba-6537112b3416 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:11:47.351924 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4face12f-1584-46ba-acba-6537112b3416 service nova] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:11:47.356594 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfbddd4-1d68-4225-816f-442acb4c13e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:11:47.357233 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfbddd4-1d68-4225-816f-442acb4c13e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:11:47.357516 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfbddd4-1d68-4225-816f-442acb4c13e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:11:47.357826 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfbddd4-1d68-4225-816f-442acb4c13e5 admin admin] PATH_INFO: `/v3/users/ca46ccf45ef54624880d68d3f7091617` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:11:47.358953 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfbddd4-1d68-4225-816f-442acb4c13e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca46ccf45ef54624880d68d3f7091617)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:11:47.360899 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfbddd4-1d68-4225-816f-442acb4c13e5 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:11:47.414331 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-4bfbddd4-1d68-4225-816f-442acb4c13e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca46ccf45ef54624880d68d3f7091617'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:11:47.415243 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-4bfbddd4-1d68-4225-816f-442acb4c13e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca46ccf45ef54624880d68d3f7091617'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:11:47.428034 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-4bfbddd4-1d68-4225-816f-442acb4c13e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca46ccf45ef54624880d68d3f7091617'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:11:47.436419 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-4bfbddd4-1d68-4225-816f-442acb4c13e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca46ccf45ef54624880d68d3f7091617'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:11:47.437035 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-4bfbddd4-1d68-4225-816f-442acb4c13e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca46ccf45ef54624880d68d3f7091617'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:11:47.448950 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 995/2005] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 09:11:47 2022] DELETE /identity/v3/users/ca46ccf45ef54624880d68d3f7091617 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:11:47.453598 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea752e12-9f6f-4ef9-b514-7170ffb873f2 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:11:47.453933 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea752e12-9f6f-4ef9-b514-7170ffb873f2 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:11:47.454211 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea752e12-9f6f-4ef9-b514-7170ffb873f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:11:47.473172 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-ea752e12-9f6f-4ef9-b514-7170ffb873f2 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:11:47.474432 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-ea752e12-9f6f-4ef9-b514-7170ffb873f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:11:47.536492 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1011/2006] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:11:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:11:47.547391 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfbddd4-1d68-4225-816f-442acb4c13e5 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:11:47.548828 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-4bfbddd4-1d68-4225-816f-442acb4c13e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:11:47.605771 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfbddd4-1d68-4225-816f-442acb4c13e5 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:11:47.609972 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fe9205-a958-4383-a8ed-acd72a6a98eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:11:47.610585 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fe9205-a958-4383-a8ed-acd72a6a98eb service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:11:47.610870 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fe9205-a958-4383-a8ed-acd72a6a98eb service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:11:47.611147 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fe9205-a958-4383-a8ed-acd72a6a98eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:11:47.620392 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fe9205-a958-4383-a8ed-acd72a6a98eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:11:47.622442 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fe9205-a958-4383-a8ed-acd72a6a98eb service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:11:47.632026 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 996/2007] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:11:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:11:47.756049 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea752e12-9f6f-4ef9-b514-7170ffb873f2 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:11:47.757773 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-ea752e12-9f6f-4ef9-b514-7170ffb873f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:11:47.861967 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea752e12-9f6f-4ef9-b514-7170ffb873f2 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:11:47.867581 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e4216c-be4a-41d3-b411-5c812abe076d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0a8af5cadc4cc1ab73e8eb2a20bb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:11:47.868194 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e4216c-be4a-41d3-b411-5c812abe076d service nova] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:11:47.868477 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e4216c-be4a-41d3-b411-5c812abe076d service nova] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:11:47.868756 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e4216c-be4a-41d3-b411-5c812abe076d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:11:47.870523 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-17e4216c-be4a-41d3-b411-5c812abe076d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:11:47.882478 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e4216c-be4a-41d3-b411-5c812abe076d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:11:47.884060 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e4216c-be4a-41d3-b411-5c812abe076d service nova] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:11:47.894746 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1012/2008] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:11:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:11:47.933385 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fe9205-a958-4383-a8ed-acd72a6a98eb service neutron] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:11:47.935061 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-23fe9205-a958-4383-a8ed-acd72a6a98eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:11:47.956402 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fe9205-a958-4383-a8ed-acd72a6a98eb service neutron] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:11:47.966111 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4ec9bd-e660-47d1-9f1e-62a10762317f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:11:47.967127 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ec9bd-e660-47d1-9f1e-62a10762317f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:11:47.967698 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ec9bd-e660-47d1-9f1e-62a10762317f admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:11:47.968096 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ec9bd-e660-47d1-9f1e-62a10762317f admin admin] PATH_INFO: `/v3/projects/41425b7af0c54491be7d15b42790dee0` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:11:47.969346 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4ec9bd-e660-47d1-9f1e-62a10762317f admin admin] RBAC: Authorizing `identity:delete_project(project_id=41425b7af0c54491be7d15b42790dee0)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:11:47.971861 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4ec9bd-e660-47d1-9f1e-62a10762317f admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:11:47.990090 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:11:48.028768 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-db4ec9bd-e660-47d1-9f1e-62a10762317f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41425b7af0c54491be7d15b42790dee0', 'request_id': 'req-db4ec9bd-e660-47d1-9f1e-62a10762317f'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:11:48.035150 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-db4ec9bd-e660-47d1-9f1e-62a10762317f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41425b7af0c54491be7d15b42790dee0', 'request_id': 'req-db4ec9bd-e660-47d1-9f1e-62a10762317f'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:11:48.042375 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-db4ec9bd-e660-47d1-9f1e-62a10762317f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41425b7af0c54491be7d15b42790dee0', 'request_id': 'req-db4ec9bd-e660-47d1-9f1e-62a10762317f'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:11:48.053544 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-db4ec9bd-e660-47d1-9f1e-62a10762317f admin admin] The token cache is being invalidate because project 41425b7af0c54491be7d15b42790dee0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 09:11:48.053869 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-db4ec9bd-e660-47d1-9f1e-62a10762317f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:11:48.054489 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-db4ec9bd-e660-47d1-9f1e-62a10762317f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:11:48.055969 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 997/2009] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 09:11:47 2022] DELETE /identity/v3/projects/41425b7af0c54491be7d15b42790dee0 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:11:48.062266 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f29130-1808-4a0f-9e27-085c2b6a4153 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:11:48.062551 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f29130-1808-4a0f-9e27-085c2b6a4153 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:11:48.062826 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f29130-1808-4a0f-9e27-085c2b6a4153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:11:48.081942 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-65f29130-1808-4a0f-9e27-085c2b6a4153 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:11:48.083129 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-65f29130-1808-4a0f-9e27-085c2b6a4153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:11:48.133113 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:11:48.143372 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1013/2010] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:11:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:11:48.151250 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c7b4e-7959-4d63-a52b-a8783506c73e None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:11:48.151539 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c7b4e-7959-4d63-a52b-a8783506c73e None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:11:48.151813 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c7b4e-7959-4d63-a52b-a8783506c73e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:11:48.173071 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-9f1c7b4e-7959-4d63-a52b-a8783506c73e None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:11:48.174353 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-9f1c7b4e-7959-4d63-a52b-a8783506c73e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:11:48.182217 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 998/2011] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:11:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:11:48.186263 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f29130-1808-4a0f-9e27-085c2b6a4153 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:11:48.195723 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f29130-1808-4a0f-9e27-085c2b6a4153 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:11:48.200803 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be92407-96b6-47bd-a588-6e9e15295561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:11:48.201394 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be92407-96b6-47bd-a588-6e9e15295561 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:11:48.201708 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be92407-96b6-47bd-a588-6e9e15295561 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:11:48.201985 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be92407-96b6-47bd-a588-6e9e15295561 admin admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:11:48.202626 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be92407-96b6-47bd-a588-6e9e15295561 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 09:11:48.202931 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be92407-96b6-47bd-a588-6e9e15295561 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:11:48.204521 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be92407-96b6-47bd-a588-6e9e15295561 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:11:48.216381 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1014/2012] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 09:11:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 09:11:48.220563 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1c7b4e-7959-4d63-a52b-a8783506c73e None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:11:48.229850 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1c7b4e-7959-4d63-a52b-a8783506c73e None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:11:48.234695 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cf4757-a66d-4345-ab7b-e46500988055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:11:48.235278 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf4757-a66d-4345-ab7b-e46500988055 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:11:48.235554 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf4757-a66d-4345-ab7b-e46500988055 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:11:48.235824 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf4757-a66d-4345-ab7b-e46500988055 admin admin] PATH_INFO: `/v3/projects` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:11:48.236440 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cf4757-a66d-4345-ab7b-e46500988055 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:11:48.238381 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cf4757-a66d-4345-ab7b-e46500988055 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:11:48.274834 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 999/2013] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 09:11:48 2022] POST /identity/v3/projects => generated 397 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:11:48.279853 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be92407-96b6-47bd-a588-6e9e15295561 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:11:48.289889 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be92407-96b6-47bd-a588-6e9e15295561 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:11:48.348743 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ee34e8-4256-4f2e-a7d9-3acbd2fd9d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:11:48.349358 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ee34e8-4256-4f2e-a7d9-3acbd2fd9d97 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:11:48.349667 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ee34e8-4256-4f2e-a7d9-3acbd2fd9d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:11:48.349973 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ee34e8-4256-4f2e-a7d9-3acbd2fd9d97 admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:11:48.350616 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ee34e8-4256-4f2e-a7d9-3acbd2fd9d97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:11:48.352513 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ee34e8-4256-4f2e-a7d9-3acbd2fd9d97 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:11:48.373025 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1015/2014] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 09:11:48 2022] POST /identity/v3/users => generated 299 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:11:48.377199 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cf4757-a66d-4345-ab7b-e46500988055 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:11:48.388197 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cf4757-a66d-4345-ab7b-e46500988055 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:11:48.392845 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64400e4-cd57-4ee0-8119-af9652b1168b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:11:48.393445 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64400e4-cd57-4ee0-8119-af9652b1168b admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:11:48.393762 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64400e4-cd57-4ee0-8119-af9652b1168b admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:11:48.394036 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64400e4-cd57-4ee0-8119-af9652b1168b admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:11:48.394716 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64400e4-cd57-4ee0-8119-af9652b1168b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:11:48.396320 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64400e4-cd57-4ee0-8119-af9652b1168b admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:11:48.406923 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1000/2015] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:11:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:11:48.410872 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ee34e8-4256-4f2e-a7d9-3acbd2fd9d97 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:11:48.420847 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ee34e8-4256-4f2e-a7d9-3acbd2fd9d97 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:11:48.425416 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fd7837-437d-454a-b354-dfb511f03712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:11:48.426040 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd7837-437d-454a-b354-dfb511f03712 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:11:48.426322 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd7837-437d-454a-b354-dfb511f03712 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:11:48.426594 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd7837-437d-454a-b354-dfb511f03712 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:11:48.427226 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fd7837-437d-454a-b354-dfb511f03712 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:11:48.428851 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fd7837-437d-454a-b354-dfb511f03712 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:11:48.439563 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1016/2016] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:11:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:11:48.443478 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64400e4-cd57-4ee0-8119-af9652b1168b admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:11:48.453715 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64400e4-cd57-4ee0-8119-af9652b1168b admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:11:48.458242 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751f6882-8025-4bac-b93e-58852f186019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:11:48.458878 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f6882-8025-4bac-b93e-58852f186019 admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:11:48.459157 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f6882-8025-4bac-b93e-58852f186019 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:11:48.459432 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f6882-8025-4bac-b93e-58852f186019 admin admin] PATH_INFO: `/v3/projects/031beaae6e144a7e923be1193b7cb4c8/users/de5f56dc1d614f11b327170320a64e31/roles/3248f0268738489b883041851d681bb7` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:11:48.474167 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751f6882-8025-4bac-b93e-58852f186019 admin admin] RBAC: Authorizing `identity:create_grant(project_id=031beaae6e144a7e923be1193b7cb4c8, user_id=de5f56dc1d614f11b327170320a64e31, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:11:48.476794 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751f6882-8025-4bac-b93e-58852f186019 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:11:48.489240 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1001/2017] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:11:48 2022] PUT /identity/v3/projects/031beaae6e144a7e923be1193b7cb4c8/users/de5f56dc1d614f11b327170320a64e31/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:11:48.500744 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eae692-09ce-4fc3-8bb7-fbb45f36ea89 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:11:48.501035 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eae692-09ce-4fc3-8bb7-fbb45f36ea89 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:11:48.501316 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eae692-09ce-4fc3-8bb7-fbb45f36ea89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:11:48.529538 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-32eae692-09ce-4fc3-8bb7-fbb45f36ea89 None None] MFA Rules not processed for user `de5f56dc1d614f11b327170320a64e31`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:11:48.530744 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-32eae692-09ce-4fc3-8bb7-fbb45f36ea89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:11:48.593370 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1017/2018] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:11:48 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:11:58.485600 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=48,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:11:58.590970 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=73,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:12:33.315678 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b383a62-de51-4eb4-bdad-365f9d8696a7 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:12:33.316237 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b383a62-de51-4eb4-bdad-365f9d8696a7 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:12:33.316237 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b383a62-de51-4eb4-bdad-365f9d8696a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:12:33.342959 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-2b383a62-de51-4eb4-bdad-365f9d8696a7 None None] MFA Rules not processed for user `a5d85f7efca040cc81253dfbf0de7c42`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:12:33.344175 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-2b383a62-de51-4eb4-bdad-365f9d8696a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:12:33.402848 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1002/2019] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:12:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:12:33.513577 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eae692-09ce-4fc3-8bb7-fbb45f36ea89 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:12:33.515547 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-32eae692-09ce-4fc3-8bb7-fbb45f36ea89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:12:33.574532 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eae692-09ce-4fc3-8bb7-fbb45f36ea89 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:12:33.580487 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb142458-a499-4ff6-93fb-88a1244a4aac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:12:33.581878 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb142458-a499-4ff6-93fb-88a1244a4aac service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:12:33.582845 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb142458-a499-4ff6-93fb-88a1244a4aac service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:12:33.583165 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb142458-a499-4ff6-93fb-88a1244a4aac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:12:33.593391 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb142458-a499-4ff6-93fb-88a1244a4aac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:12:33.594882 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb142458-a499-4ff6-93fb-88a1244a4aac service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:12:33.604702 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1018/2020] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:12:33 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:12:33.718616 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b383a62-de51-4eb4-bdad-365f9d8696a7 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:12:33.720045 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-2b383a62-de51-4eb4-bdad-365f9d8696a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:12:33.777531 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b383a62-de51-4eb4-bdad-365f9d8696a7 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:12:33.781171 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e21f2c-2046-4b34-a46d-51c1312f5462 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:12:33.781854 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e21f2c-2046-4b34-a46d-51c1312f5462 service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:12:33.782132 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e21f2c-2046-4b34-a46d-51c1312f5462 service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:12:33.782427 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e21f2c-2046-4b34-a46d-51c1312f5462 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:12:33.784094 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-e7e21f2c-2046-4b34-a46d-51c1312f5462 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:12:33.794193 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e21f2c-2046-4b34-a46d-51c1312f5462 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:12:33.795612 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e21f2c-2046-4b34-a46d-51c1312f5462 service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:12:33.803355 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1003/2021] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:12:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:12:34.122951 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb142458-a499-4ff6-93fb-88a1244a4aac service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:12:34.124327 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-fb142458-a499-4ff6-93fb-88a1244a4aac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:12:34.194585 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb142458-a499-4ff6-93fb-88a1244a4aac service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:12:34.200075 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624bdf40-6d18-4447-976c-a3b9c30c4494 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38986ca2612c4fc59d960cd693834f9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:12:34.200702 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624bdf40-6d18-4447-976c-a3b9c30c4494 service placement] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:12:34.200984 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624bdf40-6d18-4447-976c-a3b9c30c4494 service placement] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:12:34.201261 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624bdf40-6d18-4447-976c-a3b9c30c4494 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:12:34.203012 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-624bdf40-6d18-4447-976c-a3b9c30c4494 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:12:34.214170 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624bdf40-6d18-4447-976c-a3b9c30c4494 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:12:34.216353 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624bdf40-6d18-4447-976c-a3b9c30c4494 service placement] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:12:34.225988 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1019/2022] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:12:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 09:12:35.912266 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e21f2c-2046-4b34-a46d-51c1312f5462 service neutron] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:12:35.921301 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e21f2c-2046-4b34-a46d-51c1312f5462 service neutron] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:12:35.925004 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff839c9-1289-432d-a038-9276a735d6b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:12:35.925631 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff839c9-1289-432d-a038-9276a735d6b0 service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:12:35.925952 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff839c9-1289-432d-a038-9276a735d6b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:12:35.926228 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff839c9-1289-432d-a038-9276a735d6b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:12:35.927948 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-4ff839c9-1289-432d-a038-9276a735d6b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:12:35.938878 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff839c9-1289-432d-a038-9276a735d6b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:12:35.940333 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff839c9-1289-432d-a038-9276a735d6b0 service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:12:35.948867 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1004/2023] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:12:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:12:42.877822 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624bdf40-6d18-4447-976c-a3b9c30c4494 service placement] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:12:42.879438 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-624bdf40-6d18-4447-976c-a3b9c30c4494 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:12:42.954794 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624bdf40-6d18-4447-976c-a3b9c30c4494 service placement] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:12:42.960095 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631727d7-4f2c-4e6e-b989-5023d462914a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0a8af5cadc4cc1ab73e8eb2a20bb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:12:42.960698 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631727d7-4f2c-4e6e-b989-5023d462914a service nova] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:12:42.960977 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631727d7-4f2c-4e6e-b989-5023d462914a service nova] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:12:42.961249 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631727d7-4f2c-4e6e-b989-5023d462914a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:12:42.962941 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-631727d7-4f2c-4e6e-b989-5023d462914a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:12:42.975150 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631727d7-4f2c-4e6e-b989-5023d462914a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:12:42.976878 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631727d7-4f2c-4e6e-b989-5023d462914a service nova] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:12:42.986946 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1020/2024] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:12:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:12:45.945031 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=60 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:12:52.982670 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=92 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:13:55.923253 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff839c9-1289-432d-a038-9276a735d6b0 service neutron] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:13:55.924641 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-4ff839c9-1289-432d-a038-9276a735d6b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:13:56.008510 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff839c9-1289-432d-a038-9276a735d6b0 service neutron] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:13:56.013725 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cbe2f7-d522-4e60-a656-cb515eb5d6c5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38986ca2612c4fc59d960cd693834f9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:13:56.014331 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cbe2f7-d522-4e60-a656-cb515eb5d6c5 service placement] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:13:56.014612 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cbe2f7-d522-4e60-a656-cb515eb5d6c5 service placement] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:13:56.014887 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cbe2f7-d522-4e60-a656-cb515eb5d6c5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:13:56.016601 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-e7cbe2f7-d522-4e60-a656-cb515eb5d6c5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:13:56.027893 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cbe2f7-d522-4e60-a656-cb515eb5d6c5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:13:56.030131 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cbe2f7-d522-4e60-a656-cb515eb5d6c5 service placement] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:13:56.039675 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1005/2025] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:13:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:14:06.035566 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=37 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:14:33.712895 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631727d7-4f2c-4e6e-b989-5023d462914a service nova] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:14:33.732189 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631727d7-4f2c-4e6e-b989-5023d462914a service nova] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:14:33.736470 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50719d7c-fb87-440b-b013-77a64077b1a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:14:33.736967 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50719d7c-fb87-440b-b013-77a64077b1a1 service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:14:33.737247 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50719d7c-fb87-440b-b013-77a64077b1a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:14:33.737521 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50719d7c-fb87-440b-b013-77a64077b1a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:14:33.739337 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-50719d7c-fb87-440b-b013-77a64077b1a1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:14:33.772291 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50719d7c-fb87-440b-b013-77a64077b1a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:14:33.773738 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50719d7c-fb87-440b-b013-77a64077b1a1 service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:14:33.794469 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1021/2026] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:14:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:14:33.972971 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cbe2f7-d522-4e60-a656-cb515eb5d6c5 service placement] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:14:33.974462 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-e7cbe2f7-d522-4e60-a656-cb515eb5d6c5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:14:33.995834 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cbe2f7-d522-4e60-a656-cb515eb5d6c5 service placement] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:14:33.999500 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb522c36-c380-46d8-b5d4-3df5e4cf70a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:14:34.000129 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb522c36-c380-46d8-b5d4-3df5e4cf70a2 service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:14:34.000416 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb522c36-c380-46d8-b5d4-3df5e4cf70a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:14:34.000691 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb522c36-c380-46d8-b5d4-3df5e4cf70a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:14:34.002425 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-cb522c36-c380-46d8-b5d4-3df5e4cf70a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:14:34.064124 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb522c36-c380-46d8-b5d4-3df5e4cf70a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:14:34.065557 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb522c36-c380-46d8-b5d4-3df5e4cf70a2 service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:14:34.075074 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1006/2027] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:14:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:14:37.135286 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50719d7c-fb87-440b-b013-77a64077b1a1 service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:14:37.146898 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50719d7c-fb87-440b-b013-77a64077b1a1 service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:14:37.155361 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df54735-b9b5-49d3-893a-a01f1eff5f22 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38986ca2612c4fc59d960cd693834f9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:14:37.155986 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df54735-b9b5-49d3-893a-a01f1eff5f22 service placement] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:14:37.157314 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df54735-b9b5-49d3-893a-a01f1eff5f22 service placement] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:14:37.157627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df54735-b9b5-49d3-893a-a01f1eff5f22 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:14:37.159711 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-2df54735-b9b5-49d3-893a-a01f1eff5f22 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:14:37.175080 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df54735-b9b5-49d3-893a-a01f1eff5f22 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:14:37.176767 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df54735-b9b5-49d3-893a-a01f1eff5f22 service placement] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:14:37.188692 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1022/2028] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:14:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:14:37.200447 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb522c36-c380-46d8-b5d4-3df5e4cf70a2 service neutron] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:14:37.202550 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-cb522c36-c380-46d8-b5d4-3df5e4cf70a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:14:37.215292 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb522c36-c380-46d8-b5d4-3df5e4cf70a2 service neutron] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:14:37.220126 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037a7acf-f2bf-472a-a979-bac9bf7f57b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:14:37.220775 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a7acf-f2bf-472a-a979-bac9bf7f57b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:14:37.221056 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a7acf-f2bf-472a-a979-bac9bf7f57b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:14:37.221483 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a7acf-f2bf-472a-a979-bac9bf7f57b7 admin admin] PATH_INFO: `/v3/users/388ca829d87348bd9b524fd9a80e578b` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:14:37.222882 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037a7acf-f2bf-472a-a979-bac9bf7f57b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=388ca829d87348bd9b524fd9a80e578b)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:14:37.224847 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037a7acf-f2bf-472a-a979-bac9bf7f57b7 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:14:37.283506 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-037a7acf-f2bf-472a-a979-bac9bf7f57b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '388ca829d87348bd9b524fd9a80e578b'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:14:37.284160 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-037a7acf-f2bf-472a-a979-bac9bf7f57b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '388ca829d87348bd9b524fd9a80e578b'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:14:37.296317 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-037a7acf-f2bf-472a-a979-bac9bf7f57b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '388ca829d87348bd9b524fd9a80e578b'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:14:37.303067 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-037a7acf-f2bf-472a-a979-bac9bf7f57b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '388ca829d87348bd9b524fd9a80e578b'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:14:37.303725 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-037a7acf-f2bf-472a-a979-bac9bf7f57b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '388ca829d87348bd9b524fd9a80e578b'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:14:37.316151 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1007/2029] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 09:14:37 2022] DELETE /identity/v3/users/388ca829d87348bd9b524fd9a80e578b => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:14:37.482518 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df54735-b9b5-49d3-893a-a01f1eff5f22 service placement] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:14:37.495173 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df54735-b9b5-49d3-893a-a01f1eff5f22 service placement] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:14:37.552899 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e08004-94d5-4614-ba43-13f2b14cb72f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:14:37.553530 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e08004-94d5-4614-ba43-13f2b14cb72f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:14:37.553850 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e08004-94d5-4614-ba43-13f2b14cb72f admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:14:37.554131 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e08004-94d5-4614-ba43-13f2b14cb72f admin admin] PATH_INFO: `/v3/projects/23e3a95b1487450cad043a820d321f26` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:14:37.555241 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e08004-94d5-4614-ba43-13f2b14cb72f admin admin] RBAC: Authorizing `identity:delete_project(project_id=23e3a95b1487450cad043a820d321f26)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:14:37.557169 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e08004-94d5-4614-ba43-13f2b14cb72f admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:14:37.603657 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-35e08004-94d5-4614-ba43-13f2b14cb72f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23e3a95b1487450cad043a820d321f26', 'request_id': 'req-35e08004-94d5-4614-ba43-13f2b14cb72f'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:14:37.609701 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-35e08004-94d5-4614-ba43-13f2b14cb72f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23e3a95b1487450cad043a820d321f26', 'request_id': 'req-35e08004-94d5-4614-ba43-13f2b14cb72f'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:14:37.617317 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-35e08004-94d5-4614-ba43-13f2b14cb72f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23e3a95b1487450cad043a820d321f26', 'request_id': 'req-35e08004-94d5-4614-ba43-13f2b14cb72f'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:14:37.627441 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-35e08004-94d5-4614-ba43-13f2b14cb72f admin admin] The token cache is being invalidate because project 23e3a95b1487450cad043a820d321f26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 09:14:37.627728 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-35e08004-94d5-4614-ba43-13f2b14cb72f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:14:37.628353 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-35e08004-94d5-4614-ba43-13f2b14cb72f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:14:37.629846 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1023/2030] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 09:14:37 2022] DELETE /identity/v3/projects/23e3a95b1487450cad043a820d321f26 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:14:37.635695 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8592c505-9eab-43d0-a8e7-3e8dea000f74 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:14:37.635981 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8592c505-9eab-43d0-a8e7-3e8dea000f74 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:14:37.636259 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8592c505-9eab-43d0-a8e7-3e8dea000f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:14:37.656470 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-8592c505-9eab-43d0-a8e7-3e8dea000f74 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:14:37.657657 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-8592c505-9eab-43d0-a8e7-3e8dea000f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:14:37.708744 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:14:37.719576 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1008/2031] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:14:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:14:37.727512 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d84639a-ed30-482a-addf-e6b9a06d599e None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:14:37.727738 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d84639a-ed30-482a-addf-e6b9a06d599e None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:14:37.728008 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d84639a-ed30-482a-addf-e6b9a06d599e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:14:37.747928 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-8d84639a-ed30-482a-addf-e6b9a06d599e None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:14:37.749737 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-8d84639a-ed30-482a-addf-e6b9a06d599e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:14:37.757974 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1024/2032] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:14:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:14:37.762040 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8592c505-9eab-43d0-a8e7-3e8dea000f74 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:14:37.772240 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8592c505-9eab-43d0-a8e7-3e8dea000f74 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:14:37.776900 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea667d20-cb8a-4c42-8ae1-46b0f6ecc88d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:14:37.777541 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea667d20-cb8a-4c42-8ae1-46b0f6ecc88d admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:14:37.777888 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea667d20-cb8a-4c42-8ae1-46b0f6ecc88d admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:14:37.778172 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea667d20-cb8a-4c42-8ae1-46b0f6ecc88d admin admin] PATH_INFO: `/v3/domains` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:14:37.778822 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea667d20-cb8a-4c42-8ae1-46b0f6ecc88d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 09:14:37.779132 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea667d20-cb8a-4c42-8ae1-46b0f6ecc88d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:14:37.780797 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea667d20-cb8a-4c42-8ae1-46b0f6ecc88d admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:14:37.792998 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1009/2033] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 09:14:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 09:14:37.797786 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d84639a-ed30-482a-addf-e6b9a06d599e None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:14:37.807237 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d84639a-ed30-482a-addf-e6b9a06d599e None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:14:37.812684 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff7cb6a-8995-4543-9f94-0769b27c28cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:14:37.813287 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff7cb6a-8995-4543-9f94-0769b27c28cc admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:14:37.813569 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff7cb6a-8995-4543-9f94-0769b27c28cc admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:14:37.813908 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff7cb6a-8995-4543-9f94-0769b27c28cc admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:14:37.814585 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff7cb6a-8995-4543-9f94-0769b27c28cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:14:37.816636 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff7cb6a-8995-4543-9f94-0769b27c28cc admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:14:37.838758 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:14:37.850766 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1025/2034] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 09:14:37 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:14:37.851589 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea667d20-cb8a-4c42-8ae1-46b0f6ecc88d admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:14:37.862689 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea667d20-cb8a-4c42-8ae1-46b0f6ecc88d admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:14:37.921411 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c450faec-3144-4492-9313-a0d42bea6ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:14:37.922043 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c450faec-3144-4492-9313-a0d42bea6ece admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:14:37.922322 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c450faec-3144-4492-9313-a0d42bea6ece admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:14:37.922593 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c450faec-3144-4492-9313-a0d42bea6ece admin admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:14:37.923608 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c450faec-3144-4492-9313-a0d42bea6ece admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:14:37.925521 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c450faec-3144-4492-9313-a0d42bea6ece admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:14:37.945376 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1010/2035] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 09:14:37 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 09:14:37.949787 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff7cb6a-8995-4543-9f94-0769b27c28cc admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:14:37.960127 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff7cb6a-8995-4543-9f94-0769b27c28cc admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:14:37.964716 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1c9c21-d730-4c2e-8149-9c2b38bbe278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:14:37.965307 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1c9c21-d730-4c2e-8149-9c2b38bbe278 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:14:37.965591 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1c9c21-d730-4c2e-8149-9c2b38bbe278 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:14:37.965900 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1c9c21-d730-4c2e-8149-9c2b38bbe278 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:14:37.966540 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1c9c21-d730-4c2e-8149-9c2b38bbe278 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:14:37.968656 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1c9c21-d730-4c2e-8149-9c2b38bbe278 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:14:37.978559 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1026/2036] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:14:37.982349 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c450faec-3144-4492-9313-a0d42bea6ece admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:14:37.992720 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c450faec-3144-4492-9313-a0d42bea6ece admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:14:37.997285 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1f064e-cf0a-4444-90e0-6b346ec85550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:14:37.997906 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1f064e-cf0a-4444-90e0-6b346ec85550 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:14:37.998191 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1f064e-cf0a-4444-90e0-6b346ec85550 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:14:37.998978 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1f064e-cf0a-4444-90e0-6b346ec85550 admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:14:37.999628 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1f064e-cf0a-4444-90e0-6b346ec85550 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:14:38.001211 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1f064e-cf0a-4444-90e0-6b346ec85550 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:14:38.012000 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1011/2037] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:14:38.016422 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1c9c21-d730-4c2e-8149-9c2b38bbe278 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:14:38.026230 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1c9c21-d730-4c2e-8149-9c2b38bbe278 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:14:38.030889 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52929714-25a6-4f2b-a3d6-60c59a0c06d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:14:38.031533 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52929714-25a6-4f2b-a3d6-60c59a0c06d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:14:38.031817 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52929714-25a6-4f2b-a3d6-60c59a0c06d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:14:38.032093 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52929714-25a6-4f2b-a3d6-60c59a0c06d2 admin admin] PATH_INFO: `/v3/projects/3308174a881f4d5e916d3a2a849f6a5b/users/ce56910a98b042b4b1543353ddfbd4f6/roles/3248f0268738489b883041851d681bb7` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:14:38.046607 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52929714-25a6-4f2b-a3d6-60c59a0c06d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3308174a881f4d5e916d3a2a849f6a5b, user_id=ce56910a98b042b4b1543353ddfbd4f6, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:14:38.049241 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52929714-25a6-4f2b-a3d6-60c59a0c06d2 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:14:38.062085 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1027/2038] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:14:38 2022] PUT /identity/v3/projects/3308174a881f4d5e916d3a2a849f6a5b/users/ce56910a98b042b4b1543353ddfbd4f6/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:14:38.071959 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29195cf8-d298-4d5d-bba7-07c47c770bd8 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:14:38.072427 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29195cf8-d298-4d5d-bba7-07c47c770bd8 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:14:38.072935 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29195cf8-d298-4d5d-bba7-07c47c770bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:14:38.100627 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-29195cf8-d298-4d5d-bba7-07c47c770bd8 None None] MFA Rules not processed for user `ce56910a98b042b4b1543353ddfbd4f6`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:14:38.101860 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-29195cf8-d298-4d5d-bba7-07c47c770bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:14:38.164703 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1012/2039] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:14:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:14:38.169491 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52929714-25a6-4f2b-a3d6-60c59a0c06d2 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:14:38.230030 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52929714-25a6-4f2b-a3d6-60c59a0c06d2 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:14:38.235163 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23047b5-781a-4473-bb9e-2987fbe70fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:14:38.235757 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23047b5-781a-4473-bb9e-2987fbe70fe3 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:14:38.236040 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23047b5-781a-4473-bb9e-2987fbe70fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:14:38.236316 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23047b5-781a-4473-bb9e-2987fbe70fe3 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:14:38.236938 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23047b5-781a-4473-bb9e-2987fbe70fe3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:14:38.238861 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23047b5-781a-4473-bb9e-2987fbe70fe3 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:14:38.264821 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1028/2040] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 09:14:38 2022] POST /identity/v3/projects => generated 373 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:14:38.268917 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29195cf8-d298-4d5d-bba7-07c47c770bd8 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:14:38.280586 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29195cf8-d298-4d5d-bba7-07c47c770bd8 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:14:38.326296 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:14:38.336441 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85298d0-1929-435b-9a32-11207e4bbf90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:14:38.337044 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85298d0-1929-435b-9a32-11207e4bbf90 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:14:38.337324 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85298d0-1929-435b-9a32-11207e4bbf90 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:14:38.337599 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85298d0-1929-435b-9a32-11207e4bbf90 admin admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:14:38.338260 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85298d0-1929-435b-9a32-11207e4bbf90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:14:38.340214 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85298d0-1929-435b-9a32-11207e4bbf90 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:14:38.363859 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1013/2041] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 09:14:38 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:14:38.367816 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23047b5-781a-4473-bb9e-2987fbe70fe3 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:14:38.379082 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23047b5-781a-4473-bb9e-2987fbe70fe3 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:14:38.387871 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bf6379-d7da-44b2-a127-9359538be713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:14:38.388495 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bf6379-d7da-44b2-a127-9359538be713 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:14:38.388778 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bf6379-d7da-44b2-a127-9359538be713 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:14:38.389581 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bf6379-d7da-44b2-a127-9359538be713 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:14:38.390261 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bf6379-d7da-44b2-a127-9359538be713 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:14:38.391857 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bf6379-d7da-44b2-a127-9359538be713 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:14:38.403001 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1029/2042] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:14:38.407835 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85298d0-1929-435b-9a32-11207e4bbf90 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:14:38.418010 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85298d0-1929-435b-9a32-11207e4bbf90 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:14:38.422574 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b979a5d-5e34-4c10-893c-32e6d8d07b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:14:38.423220 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b979a5d-5e34-4c10-893c-32e6d8d07b9c admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:14:38.423503 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b979a5d-5e34-4c10-893c-32e6d8d07b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:14:38.423780 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b979a5d-5e34-4c10-893c-32e6d8d07b9c admin admin] PATH_INFO: `/v3/projects/31538e5c592648b1b539ccde73923724/users/a7fdde37ad454ad2a570738ec8a6639f/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:14:38.438413 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b979a5d-5e34-4c10-893c-32e6d8d07b9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=31538e5c592648b1b539ccde73923724, user_id=a7fdde37ad454ad2a570738ec8a6639f, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:14:38.441109 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b979a5d-5e34-4c10-893c-32e6d8d07b9c admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:14:38.454330 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1014/2043] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:14:38 2022] PUT /identity/v3/projects/31538e5c592648b1b539ccde73923724/users/a7fdde37ad454ad2a570738ec8a6639f/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:14:38.458365 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bf6379-d7da-44b2-a127-9359538be713 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:14:38.468865 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bf6379-d7da-44b2-a127-9359538be713 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:14:38.523994 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3016a160-654f-4f04-ba6f-cd8def642981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:14:38.524873 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3016a160-654f-4f04-ba6f-cd8def642981 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:14:38.525161 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3016a160-654f-4f04-ba6f-cd8def642981 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:14:38.525437 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3016a160-654f-4f04-ba6f-cd8def642981 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:14:38.526125 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3016a160-654f-4f04-ba6f-cd8def642981 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:14:38.527708 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3016a160-654f-4f04-ba6f-cd8def642981 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:14:38.537906 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1030/2044] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:14:38.541992 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b979a5d-5e34-4c10-893c-32e6d8d07b9c admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:14:38.552848 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b979a5d-5e34-4c10-893c-32e6d8d07b9c admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:14:38.557540 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d87c6f-773f-4b58-9087-9ae6712e3975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:14:38.558224 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d87c6f-773f-4b58-9087-9ae6712e3975 admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:14:38.558508 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d87c6f-773f-4b58-9087-9ae6712e3975 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:14:38.558785 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d87c6f-773f-4b58-9087-9ae6712e3975 admin admin] PATH_INFO: `/v3/projects/31538e5c592648b1b539ccde73923724/users/a7fdde37ad454ad2a570738ec8a6639f/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:14:38.560749 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d87c6f-773f-4b58-9087-9ae6712e3975 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31538e5c592648b1b539ccde73923724, user_id=a7fdde37ad454ad2a570738ec8a6639f, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:14:38.564055 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d87c6f-773f-4b58-9087-9ae6712e3975 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:14:38.574130 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1015/2045] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:14:38 2022] PUT /identity/v3/projects/31538e5c592648b1b539ccde73923724/users/a7fdde37ad454ad2a570738ec8a6639f/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:14:38.585903 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d2d342-92e4-4eca-a515-7c560edfea1f None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:14:38.586194 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d2d342-92e4-4eca-a515-7c560edfea1f None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:14:38.586479 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d2d342-92e4-4eca-a515-7c560edfea1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:14:38.614013 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-44d2d342-92e4-4eca-a515-7c560edfea1f None None] MFA Rules not processed for user `a7fdde37ad454ad2a570738ec8a6639f`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:14:38.615218 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-44d2d342-92e4-4eca-a515-7c560edfea1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:14:38.635025 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:14:38.684504 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1031/2046] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:14:38 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:14:48.570497 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=20,keystone:INSERT=5 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:14:48.681591 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=19 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:15:38.762419 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d87c6f-773f-4b58-9087-9ae6712e3975 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:15:38.831112 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d87c6f-773f-4b58-9087-9ae6712e3975 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:15:38.835634 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaea66b-2bd8-4fe4-8633-11734633c834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:15:38.836259 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaea66b-2bd8-4fe4-8633-11734633c834 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:15:38.836545 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaea66b-2bd8-4fe4-8633-11734633c834 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:15:38.836819 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaea66b-2bd8-4fe4-8633-11734633c834 admin admin] PATH_INFO: `/v3/users/ce56910a98b042b4b1543353ddfbd4f6` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:15:38.837949 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaea66b-2bd8-4fe4-8633-11734633c834 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce56910a98b042b4b1543353ddfbd4f6)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:15:38.839868 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaea66b-2bd8-4fe4-8633-11734633c834 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:15:38.884752 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-dbaea66b-2bd8-4fe4-8633-11734633c834 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce56910a98b042b4b1543353ddfbd4f6'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:15:38.885396 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-dbaea66b-2bd8-4fe4-8633-11734633c834 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce56910a98b042b4b1543353ddfbd4f6'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:15:38.897545 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-dbaea66b-2bd8-4fe4-8633-11734633c834 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce56910a98b042b4b1543353ddfbd4f6'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:15:38.904613 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-dbaea66b-2bd8-4fe4-8633-11734633c834 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce56910a98b042b4b1543353ddfbd4f6'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:15:38.905266 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-dbaea66b-2bd8-4fe4-8633-11734633c834 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce56910a98b042b4b1543353ddfbd4f6'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:15:38.917269 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1016/2047] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 09:15:38 2022] DELETE /identity/v3/users/ce56910a98b042b4b1543353ddfbd4f6 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:15:38.922148 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf74f5-ee39-4080-9f4a-0e7842996d62 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:15:38.922620 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf74f5-ee39-4080-9f4a-0e7842996d62 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:15:38.922802 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf74f5-ee39-4080-9f4a-0e7842996d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:15:38.941643 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-1fbf74f5-ee39-4080-9f4a-0e7842996d62 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:15:38.942856 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-1fbf74f5-ee39-4080-9f4a-0e7842996d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:15:38.999829 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1032/2048] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:15:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 09:15:39.009368 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaea66b-2bd8-4fe4-8633-11734633c834 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:15:39.010803 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-dbaea66b-2bd8-4fe4-8633-11734633c834 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:15:39.067827 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaea66b-2bd8-4fe4-8633-11734633c834 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:15:39.071352 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44735af-ca17-48dd-b6ee-550729b26a51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:15:39.071950 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44735af-ca17-48dd-b6ee-550729b26a51 service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:15:39.072227 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44735af-ca17-48dd-b6ee-550729b26a51 service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:15:39.072499 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44735af-ca17-48dd-b6ee-550729b26a51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:15:39.081279 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44735af-ca17-48dd-b6ee-550729b26a51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:15:39.082731 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44735af-ca17-48dd-b6ee-550729b26a51 service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:15:39.092261 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1017/2049] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:15:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:15:39.356732 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbf74f5-ee39-4080-9f4a-0e7842996d62 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:15:39.358170 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-1fbf74f5-ee39-4080-9f4a-0e7842996d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:15:39.370234 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbf74f5-ee39-4080-9f4a-0e7842996d62 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:15:39.374798 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d74dd68-3f77-4bfe-af4e-f15408dc257f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:15:39.375753 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d74dd68-3f77-4bfe-af4e-f15408dc257f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:15:39.375753 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d74dd68-3f77-4bfe-af4e-f15408dc257f admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:15:39.375949 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d74dd68-3f77-4bfe-af4e-f15408dc257f admin admin] PATH_INFO: `/v3/projects/3308174a881f4d5e916d3a2a849f6a5b` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:15:39.377070 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d74dd68-3f77-4bfe-af4e-f15408dc257f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3308174a881f4d5e916d3a2a849f6a5b)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:15:39.379019 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d74dd68-3f77-4bfe-af4e-f15408dc257f admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:15:39.427828 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8d74dd68-3f77-4bfe-af4e-f15408dc257f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3308174a881f4d5e916d3a2a849f6a5b', 'request_id': 'req-8d74dd68-3f77-4bfe-af4e-f15408dc257f'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:15:39.434150 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8d74dd68-3f77-4bfe-af4e-f15408dc257f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3308174a881f4d5e916d3a2a849f6a5b', 'request_id': 'req-8d74dd68-3f77-4bfe-af4e-f15408dc257f'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:15:39.441157 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8d74dd68-3f77-4bfe-af4e-f15408dc257f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3308174a881f4d5e916d3a2a849f6a5b', 'request_id': 'req-8d74dd68-3f77-4bfe-af4e-f15408dc257f'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:15:39.451191 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8d74dd68-3f77-4bfe-af4e-f15408dc257f admin admin] The token cache is being invalidate because project 3308174a881f4d5e916d3a2a849f6a5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 09:15:39.451477 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8d74dd68-3f77-4bfe-af4e-f15408dc257f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:15:39.452099 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8d74dd68-3f77-4bfe-af4e-f15408dc257f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:15:39.453560 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1033/2050] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 09:15:39 2022] DELETE /identity/v3/projects/3308174a881f4d5e916d3a2a849f6a5b => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:15:39.457461 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44735af-ca17-48dd-b6ee-550729b26a51 service neutron] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:15:39.466811 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44735af-ca17-48dd-b6ee-550729b26a51 service neutron] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:15:39.485279 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:15:39.525347 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539b304b-228f-4c0b-9f88-9b06cce18515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:15:39.526015 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b304b-228f-4c0b-9f88-9b06cce18515 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:15:39.526301 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b304b-228f-4c0b-9f88-9b06cce18515 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:15:39.526576 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b304b-228f-4c0b-9f88-9b06cce18515 admin admin] PATH_INFO: `/v3/users/a7fdde37ad454ad2a570738ec8a6639f` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:15:39.527687 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539b304b-228f-4c0b-9f88-9b06cce18515 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7fdde37ad454ad2a570738ec8a6639f)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:15:39.529615 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539b304b-228f-4c0b-9f88-9b06cce18515 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:15:39.575175 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-539b304b-228f-4c0b-9f88-9b06cce18515 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7fdde37ad454ad2a570738ec8a6639f'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:15:39.575814 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-539b304b-228f-4c0b-9f88-9b06cce18515 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7fdde37ad454ad2a570738ec8a6639f'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:15:39.587411 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-539b304b-228f-4c0b-9f88-9b06cce18515 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7fdde37ad454ad2a570738ec8a6639f'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:15:39.595382 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-539b304b-228f-4c0b-9f88-9b06cce18515 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7fdde37ad454ad2a570738ec8a6639f'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:15:39.596097 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-539b304b-228f-4c0b-9f88-9b06cce18515 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7fdde37ad454ad2a570738ec8a6639f'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:15:39.608306 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1018/2051] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 09:15:39 2022] DELETE /identity/v3/users/a7fdde37ad454ad2a570738ec8a6639f => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:15:39.870379 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d74dd68-3f77-4bfe-af4e-f15408dc257f admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:15:39.871915 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-8d74dd68-3f77-4bfe-af4e-f15408dc257f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:15:39.887393 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d74dd68-3f77-4bfe-af4e-f15408dc257f admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:15:39.945053 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e252b40f-a27a-4a9f-83e7-133159995090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:15:39.945724 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e252b40f-a27a-4a9f-83e7-133159995090 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:15:39.945984 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e252b40f-a27a-4a9f-83e7-133159995090 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:15:39.946262 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e252b40f-a27a-4a9f-83e7-133159995090 admin admin] PATH_INFO: `/v3/projects/31538e5c592648b1b539ccde73923724` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:15:39.947373 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e252b40f-a27a-4a9f-83e7-133159995090 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31538e5c592648b1b539ccde73923724)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:15:39.949279 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e252b40f-a27a-4a9f-83e7-133159995090 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:15:39.979785 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:15:39.996685 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-e252b40f-a27a-4a9f-83e7-133159995090 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31538e5c592648b1b539ccde73923724', 'request_id': 'req-e252b40f-a27a-4a9f-83e7-133159995090'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:15:40.002870 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-e252b40f-a27a-4a9f-83e7-133159995090 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31538e5c592648b1b539ccde73923724', 'request_id': 'req-e252b40f-a27a-4a9f-83e7-133159995090'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:15:40.009777 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-e252b40f-a27a-4a9f-83e7-133159995090 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31538e5c592648b1b539ccde73923724', 'request_id': 'req-e252b40f-a27a-4a9f-83e7-133159995090'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:15:40.020844 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-e252b40f-a27a-4a9f-83e7-133159995090 admin admin] The token cache is being invalidate because project 31538e5c592648b1b539ccde73923724 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 09:15:40.021131 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-e252b40f-a27a-4a9f-83e7-133159995090 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:15:40.021792 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-e252b40f-a27a-4a9f-83e7-133159995090 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:15:40.023269 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1034/2052] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 09:15:39 2022] DELETE /identity/v3/projects/31538e5c592648b1b539ccde73923724 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:15:40.028302 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef66302-21e8-4cdf-bb0a-57546fab3c43 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:15:40.028772 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef66302-21e8-4cdf-bb0a-57546fab3c43 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:15:40.028901 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef66302-21e8-4cdf-bb0a-57546fab3c43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:15:40.050246 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-8ef66302-21e8-4cdf-bb0a-57546fab3c43 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:15:40.051446 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-8ef66302-21e8-4cdf-bb0a-57546fab3c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:15:40.107909 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1019/2053] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:15:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:15:40.115836 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126a05e1-a3b5-40fb-81b6-1a768880f092 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:15:40.116062 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126a05e1-a3b5-40fb-81b6-1a768880f092 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:15:40.116337 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126a05e1-a3b5-40fb-81b6-1a768880f092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:15:40.136484 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-126a05e1-a3b5-40fb-81b6-1a768880f092 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:15:40.137696 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-126a05e1-a3b5-40fb-81b6-1a768880f092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:15:40.146128 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1035/2054] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:15:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:15:40.150162 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef66302-21e8-4cdf-bb0a-57546fab3c43 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:15:40.159764 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef66302-21e8-4cdf-bb0a-57546fab3c43 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:15:40.164175 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b30819e-330b-4420-992a-5018cdbd3a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:15:40.164767 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b30819e-330b-4420-992a-5018cdbd3a7e admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:15:40.165043 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b30819e-330b-4420-992a-5018cdbd3a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:15:40.165315 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b30819e-330b-4420-992a-5018cdbd3a7e admin admin] PATH_INFO: `/v3/domains` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:15:40.165985 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b30819e-330b-4420-992a-5018cdbd3a7e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 09:15:40.166286 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b30819e-330b-4420-992a-5018cdbd3a7e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:15:40.167879 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b30819e-330b-4420-992a-5018cdbd3a7e admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:15:40.179329 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1020/2055] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 09:15:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 09:15:40.183345 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126a05e1-a3b5-40fb-81b6-1a768880f092 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:15:40.193509 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126a05e1-a3b5-40fb-81b6-1a768880f092 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:15:40.198104 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af90608a-9b00-47cc-9aaf-ac080f0da36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:15:40.198691 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af90608a-9b00-47cc-9aaf-ac080f0da36c admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:15:40.198972 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af90608a-9b00-47cc-9aaf-ac080f0da36c admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:15:40.199252 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af90608a-9b00-47cc-9aaf-ac080f0da36c admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:15:40.199887 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af90608a-9b00-47cc-9aaf-ac080f0da36c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:15:40.201805 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af90608a-9b00-47cc-9aaf-ac080f0da36c admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:15:40.229371 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1036/2056] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 09:15:40 2022] POST /identity/v3/projects => generated 409 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:15:40.233524 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b30819e-330b-4420-992a-5018cdbd3a7e admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:15:40.243450 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b30819e-330b-4420-992a-5018cdbd3a7e admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:15:40.269427 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:15:40.303123 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b28fcd1-5caa-4ebe-a7ae-8f68904d0921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:15:40.303727 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b28fcd1-5caa-4ebe-a7ae-8f68904d0921 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:15:40.304007 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b28fcd1-5caa-4ebe-a7ae-8f68904d0921 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:15:40.304281 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b28fcd1-5caa-4ebe-a7ae-8f68904d0921 admin admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:15:40.304899 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b28fcd1-5caa-4ebe-a7ae-8f68904d0921 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:15:40.306857 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b28fcd1-5caa-4ebe-a7ae-8f68904d0921 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:15:40.327245 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1021/2057] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 09:15:40 2022] POST /identity/v3/users => generated 305 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 09:15:40.331343 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af90608a-9b00-47cc-9aaf-ac080f0da36c admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:15:40.341379 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af90608a-9b00-47cc-9aaf-ac080f0da36c admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:15:40.346556 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a32823-116c-4107-982d-14d3889a60a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:15:40.347151 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a32823-116c-4107-982d-14d3889a60a6 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:15:40.347437 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a32823-116c-4107-982d-14d3889a60a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:15:40.347719 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a32823-116c-4107-982d-14d3889a60a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:15:40.348361 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a32823-116c-4107-982d-14d3889a60a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:15:40.349989 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a32823-116c-4107-982d-14d3889a60a6 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:15:40.360332 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1037/2058] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:15:40.364103 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b28fcd1-5caa-4ebe-a7ae-8f68904d0921 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:15:40.373955 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b28fcd1-5caa-4ebe-a7ae-8f68904d0921 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:15:40.378434 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e17769-1950-473e-bc1d-c798d09de60a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:15:40.379023 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e17769-1950-473e-bc1d-c798d09de60a admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:15:40.379308 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e17769-1950-473e-bc1d-c798d09de60a admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:15:40.379584 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e17769-1950-473e-bc1d-c798d09de60a admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:15:40.380218 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e17769-1950-473e-bc1d-c798d09de60a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:15:40.381882 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e17769-1950-473e-bc1d-c798d09de60a admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:15:40.392419 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1022/2059] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:15:40.396532 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a32823-116c-4107-982d-14d3889a60a6 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:15:40.407084 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a32823-116c-4107-982d-14d3889a60a6 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:15:40.411774 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3ffcb7-f79b-46a9-a3dd-2c90e9f104d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:15:40.412428 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ffcb7-f79b-46a9-a3dd-2c90e9f104d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:15:40.412712 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ffcb7-f79b-46a9-a3dd-2c90e9f104d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:15:40.412992 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ffcb7-f79b-46a9-a3dd-2c90e9f104d6 admin admin] PATH_INFO: `/v3/projects/b3c3719c7c56456cab90d697730dbaaf/users/7c1d5294efe4497eb2fe8b6b6982b5fd/roles/3248f0268738489b883041851d681bb7` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:15:40.428210 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3ffcb7-f79b-46a9-a3dd-2c90e9f104d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3c3719c7c56456cab90d697730dbaaf, user_id=7c1d5294efe4497eb2fe8b6b6982b5fd, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:15:40.430888 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3ffcb7-f79b-46a9-a3dd-2c90e9f104d6 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:15:40.443416 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1038/2060] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:15:40 2022] PUT /identity/v3/projects/b3c3719c7c56456cab90d697730dbaaf/users/7c1d5294efe4497eb2fe8b6b6982b5fd/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:15:40.454177 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c657ce1e-8fc3-43f1-9f0c-fa1f1d6b24c6 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:15:40.454467 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c657ce1e-8fc3-43f1-9f0c-fa1f1d6b24c6 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:15:40.454745 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c657ce1e-8fc3-43f1-9f0c-fa1f1d6b24c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:15:40.482322 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-c657ce1e-8fc3-43f1-9f0c-fa1f1d6b24c6 None None] MFA Rules not processed for user `7c1d5294efe4497eb2fe8b6b6982b5fd`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:15:40.483527 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-c657ce1e-8fc3-43f1-9f0c-fa1f1d6b24c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:15:40.546324 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1023/2061] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:15:40 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:15:40.550832 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3ffcb7-f79b-46a9-a3dd-2c90e9f104d6 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:15:40.611171 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3ffcb7-f79b-46a9-a3dd-2c90e9f104d6 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:15:40.615805 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0d90e8-8fa7-407d-a844-877cce4f0f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:15:40.616403 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d90e8-8fa7-407d-a844-877cce4f0f9c admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:15:40.616686 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d90e8-8fa7-407d-a844-877cce4f0f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:15:40.616962 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d90e8-8fa7-407d-a844-877cce4f0f9c admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:15:40.617590 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0d90e8-8fa7-407d-a844-877cce4f0f9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:15:40.619505 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0d90e8-8fa7-407d-a844-877cce4f0f9c admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:15:40.646913 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1039/2062] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 09:15:40 2022] POST /identity/v3/projects => generated 407 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:15:40.650821 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c657ce1e-8fc3-43f1-9f0c-fa1f1d6b24c6 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:15:40.662017 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c657ce1e-8fc3-43f1-9f0c-fa1f1d6b24c6 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:15:40.716019 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3c7b55-9771-4f18-9d30-f6e96dc6f7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:15:40.716621 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3c7b55-9771-4f18-9d30-f6e96dc6f7ed admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:15:40.716900 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3c7b55-9771-4f18-9d30-f6e96dc6f7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:15:40.717173 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3c7b55-9771-4f18-9d30-f6e96dc6f7ed admin admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:15:40.718439 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3c7b55-9771-4f18-9d30-f6e96dc6f7ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:15:40.720369 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3c7b55-9771-4f18-9d30-f6e96dc6f7ed admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:15:40.740657 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1024/2063] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 09:15:40 2022] POST /identity/v3/users => generated 310 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:15:40.744663 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0d90e8-8fa7-407d-a844-877cce4f0f9c admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:15:40.755490 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0d90e8-8fa7-407d-a844-877cce4f0f9c admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:15:40.760124 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d726c2f1-e5a1-47ef-aec1-3d64739dd5e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:15:40.760728 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d726c2f1-e5a1-47ef-aec1-3d64739dd5e3 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:15:40.761011 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d726c2f1-e5a1-47ef-aec1-3d64739dd5e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:15:40.761288 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d726c2f1-e5a1-47ef-aec1-3d64739dd5e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:15:40.762023 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d726c2f1-e5a1-47ef-aec1-3d64739dd5e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:15:40.763639 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d726c2f1-e5a1-47ef-aec1-3d64739dd5e3 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:15:40.774228 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1040/2064] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:15:40.778301 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3c7b55-9771-4f18-9d30-f6e96dc6f7ed admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:15:40.788125 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3c7b55-9771-4f18-9d30-f6e96dc6f7ed admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:15:40.792596 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8ec2ef-8e18-48b0-98ce-67fe5380167a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:15:40.793501 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8ec2ef-8e18-48b0-98ce-67fe5380167a admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:15:40.793812 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8ec2ef-8e18-48b0-98ce-67fe5380167a admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:15:40.794083 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8ec2ef-8e18-48b0-98ce-67fe5380167a admin admin] PATH_INFO: `/v3/projects/34a1a128d6b84b1a9529754bc3b2aa49/users/1c49a29ddcb84fb3b274fbc0fd8e2928/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:15:40.808850 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8ec2ef-8e18-48b0-98ce-67fe5380167a admin admin] RBAC: Authorizing `identity:create_grant(project_id=34a1a128d6b84b1a9529754bc3b2aa49, user_id=1c49a29ddcb84fb3b274fbc0fd8e2928, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:15:40.812160 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8ec2ef-8e18-48b0-98ce-67fe5380167a admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:15:40.824540 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1025/2065] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:15:40 2022] PUT /identity/v3/projects/34a1a128d6b84b1a9529754bc3b2aa49/users/1c49a29ddcb84fb3b274fbc0fd8e2928/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:15:40.828544 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d726c2f1-e5a1-47ef-aec1-3d64739dd5e3 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:15:40.838410 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d726c2f1-e5a1-47ef-aec1-3d64739dd5e3 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:15:40.859157 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:15:40.895695 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a50264-314a-4d05-8eab-659235117b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:15:40.896295 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a50264-314a-4d05-8eab-659235117b24 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:15:40.896579 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a50264-314a-4d05-8eab-659235117b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:15:40.896857 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a50264-314a-4d05-8eab-659235117b24 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:15:40.897544 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a50264-314a-4d05-8eab-659235117b24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:15:40.899111 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a50264-314a-4d05-8eab-659235117b24 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:15:40.909088 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1041/2066] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:15:40.913150 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8ec2ef-8e18-48b0-98ce-67fe5380167a admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:15:40.918699 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:15:40.925438 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8ec2ef-8e18-48b0-98ce-67fe5380167a admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:15:40.930242 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c080338-b475-4aca-bb0d-c0a31fa1d608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:15:40.930885 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c080338-b475-4aca-bb0d-c0a31fa1d608 admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:15:40.931162 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c080338-b475-4aca-bb0d-c0a31fa1d608 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:15:40.931438 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c080338-b475-4aca-bb0d-c0a31fa1d608 admin admin] PATH_INFO: `/v3/projects/34a1a128d6b84b1a9529754bc3b2aa49/users/1c49a29ddcb84fb3b274fbc0fd8e2928/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:15:40.933290 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c080338-b475-4aca-bb0d-c0a31fa1d608 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34a1a128d6b84b1a9529754bc3b2aa49, user_id=1c49a29ddcb84fb3b274fbc0fd8e2928, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:15:40.935966 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c080338-b475-4aca-bb0d-c0a31fa1d608 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:15:40.946017 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1026/2067] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:15:40 2022] PUT /identity/v3/projects/34a1a128d6b84b1a9529754bc3b2aa49/users/1c49a29ddcb84fb3b274fbc0fd8e2928/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:15:40.957258 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d04233-a274-4c7d-95a9-ad31a1f6b388 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:15:40.957752 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d04233-a274-4c7d-95a9-ad31a1f6b388 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:15:40.957862 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d04233-a274-4c7d-95a9-ad31a1f6b388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:15:40.986266 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-f6d04233-a274-4c7d-95a9-ad31a1f6b388 None None] MFA Rules not processed for user `1c49a29ddcb84fb3b274fbc0fd8e2928`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:15:40.987519 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-f6d04233-a274-4c7d-95a9-ad31a1f6b388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:15:41.053596 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1042/2068] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:15:40 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:15:48.382598 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c080338-b475-4aca-bb0d-c0a31fa1d608 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:15:48.384317 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-8c080338-b475-4aca-bb0d-c0a31fa1d608 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:15:48.451436 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c080338-b475-4aca-bb0d-c0a31fa1d608 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:15:48.455593 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8071a843-1f48-4676-87f9-ccd8dabb8576 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:15:48.456193 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8071a843-1f48-4676-87f9-ccd8dabb8576 service cinder] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:15:48.456468 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8071a843-1f48-4676-87f9-ccd8dabb8576 service cinder] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:15:48.456760 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8071a843-1f48-4676-87f9-ccd8dabb8576 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:15:48.458465 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-8071a843-1f48-4676-87f9-ccd8dabb8576 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:15:48.524574 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8071a843-1f48-4676-87f9-ccd8dabb8576 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:15:48.526089 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8071a843-1f48-4676-87f9-ccd8dabb8576 service cinder] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:15:48.544396 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1027/2069] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:15:48 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:15:48.585851 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d04233-a274-4c7d-95a9-ad31a1f6b388 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:15:48.595831 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d04233-a274-4c7d-95a9-ad31a1f6b388 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:15:48.599425 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a8baf5-8775-4244-9a3a-a762ac131afc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:15:48.600027 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a8baf5-8775-4244-9a3a-a762ac131afc service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:15:48.600309 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a8baf5-8775-4244-9a3a-a762ac131afc service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:15:48.600587 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a8baf5-8775-4244-9a3a-a762ac131afc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:15:48.602403 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-36a8baf5-8775-4244-9a3a-a762ac131afc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:15:48.660460 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a8baf5-8775-4244-9a3a-a762ac131afc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:15:48.661830 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a8baf5-8775-4244-9a3a-a762ac131afc service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:15:48.672047 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1043/2070] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:15:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:15:48.768127 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8071a843-1f48-4676-87f9-ccd8dabb8576 service cinder] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:15:48.776612 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8071a843-1f48-4676-87f9-ccd8dabb8576 service cinder] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:15:48.780834 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb84b7e-27c5-443a-9f25-b3ad664d1912 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:15:48.781433 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb84b7e-27c5-443a-9f25-b3ad664d1912 service cinder] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:15:48.781744 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb84b7e-27c5-443a-9f25-b3ad664d1912 service cinder] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:15:48.782017 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb84b7e-27c5-443a-9f25-b3ad664d1912 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:15:48.783719 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-8cb84b7e-27c5-443a-9f25-b3ad664d1912 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:15:48.800401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application [None req-8cb84b7e-27c5-443a-9f25-b3ad664d1912 service cinder] Could not find project: 31538e5c592648b1b539ccde73923724.: keystone.exception.ProjectNotFound: Could not find project: 31538e5c592648b1b539ccde73923724. Oct 15 09:15:48.800401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 09:15:48.800401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 09:15:48.800401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 09:15:48.800401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 09:15:48.800401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 09:15:48.800401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 09:15:48.800401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 09:15:48.800401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 09:15:48.800401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 09:15:48.800401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 09:15:48.800401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 09:15:48.800401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Oct 15 09:15:48.800401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Oct 15 09:15:48.800401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Oct 15 09:15:48.801462 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Oct 15 09:15:48.801462 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Oct 15 09:15:48.801462 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Oct 15 09:15:48.801462 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 09:15:48.801462 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 09:15:48.801462 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Oct 15 09:15:48.801462 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Oct 15 09:15:48.801462 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 09:15:48.801462 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 09:15:48.801462 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 09:15:48.801462 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 09:15:48.801462 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 09:15:48.801462 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 09:15:48.801462 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 09:15:48.801462 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 09:15:48.801462 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 09:15:48.801462 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 09:15:48.802431 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 09:15:48.802431 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 09:15:48.802431 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 09:15:48.802431 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 09:15:48.802431 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Oct 15 09:15:48.802431 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Oct 15 09:15:48.802431 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Oct 15 09:15:48.802431 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application self._validate_token_resources() Oct 15 09:15:48.802431 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Oct 15 09:15:48.802431 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Oct 15 09:15:48.802431 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Oct 15 09:15:48.802431 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Oct 15 09:15:48.802431 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 09:15:48.802431 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 09:15:48.802431 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 09:15:48.802431 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 09:15:48.802431 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 09:15:48.803286 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 09:15:48.803286 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 09:15:48.803286 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application with Lock( Oct 15 09:15:48.803286 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 09:15:48.803286 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._enter() Oct 15 09:15:48.803286 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 09:15:48.803286 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 09:15:48.803286 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 09:15:48.803286 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.creator() Oct 15 09:15:48.803286 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 09:15:48.803286 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application created_value = creator( Oct 15 09:15:48.803286 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 09:15:48.803286 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 09:15:48.803286 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 09:15:48.803286 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 09:15:48.803286 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 09:15:48.803286 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 09:15:48.804151 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 31538e5c592648b1b539ccde73923724. Oct 15 09:15:48.804151 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: ERROR keystone.server.flask.application Oct 15 09:15:48.804151 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1028/2071] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:15:48 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 34 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 15 09:15:48.923133 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a8baf5-8775-4244-9a3a-a762ac131afc service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:15:48.931648 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a8baf5-8775-4244-9a3a-a762ac131afc service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:15:48.935231 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f68e61-ef86-4ae5-921b-495eb1791fec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:15:48.935820 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f68e61-ef86-4ae5-921b-495eb1791fec service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:15:48.936100 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f68e61-ef86-4ae5-921b-495eb1791fec service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:15:48.936372 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f68e61-ef86-4ae5-921b-495eb1791fec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:15:48.938093 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-83f68e61-ef86-4ae5-921b-495eb1791fec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:15:48.981127 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:15:49.008715 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f68e61-ef86-4ae5-921b-495eb1791fec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:15:49.010190 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f68e61-ef86-4ae5-921b-495eb1791fec service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:15:49.021391 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1044/2072] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:15:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:15:49.084037 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb84b7e-27c5-443a-9f25-b3ad664d1912 service cinder] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:15:49.093375 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb84b7e-27c5-443a-9f25-b3ad664d1912 service cinder] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:15:49.097131 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea33242f-71bf-4dcc-9571-8c0a0f513f3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:15:49.097777 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea33242f-71bf-4dcc-9571-8c0a0f513f3a service cinder] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:15:49.098063 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea33242f-71bf-4dcc-9571-8c0a0f513f3a service cinder] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:15:49.098340 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea33242f-71bf-4dcc-9571-8c0a0f513f3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:15:49.107328 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea33242f-71bf-4dcc-9571-8c0a0f513f3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:15:49.108784 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea33242f-71bf-4dcc-9571-8c0a0f513f3a service cinder] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:15:49.118168 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1029/2073] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:15:49 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:15:59.016549 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=14 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:15:59.113871 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=74,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:16:34.370281 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f68e61-ef86-4ae5-921b-495eb1791fec service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:16:34.373662 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-83f68e61-ef86-4ae5-921b-495eb1791fec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:16:34.384346 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f68e61-ef86-4ae5-921b-495eb1791fec service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:16:34.388426 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c5524c-e3ed-4e8c-9f78-e32e1f8f430d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:16:34.389035 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c5524c-e3ed-4e8c-9f78-e32e1f8f430d service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:16:34.389318 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c5524c-e3ed-4e8c-9f78-e32e1f8f430d service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:16:34.389593 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c5524c-e3ed-4e8c-9f78-e32e1f8f430d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:16:34.391500 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-27c5524c-e3ed-4e8c-9f78-e32e1f8f430d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:16:34.484886 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c5524c-e3ed-4e8c-9f78-e32e1f8f430d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:16:34.486509 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c5524c-e3ed-4e8c-9f78-e32e1f8f430d service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:16:34.505064 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1045/2074] 10.222.0.30 () {60 vars in 1396 bytes} [Sat Oct 15 09:16:34 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3250 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:16:34.602462 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea33242f-71bf-4dcc-9571-8c0a0f513f3a service cinder] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:16:34.612900 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea33242f-71bf-4dcc-9571-8c0a0f513f3a service cinder] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:16:34.616453 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7236860f-e14e-4ff8-a66f-b1cadb6496a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:16:34.617040 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7236860f-e14e-4ff8-a66f-b1cadb6496a1 service cinder] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:16:34.617318 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7236860f-e14e-4ff8-a66f-b1cadb6496a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:16:34.617588 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7236860f-e14e-4ff8-a66f-b1cadb6496a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:16:34.672946 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7236860f-e14e-4ff8-a66f-b1cadb6496a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:16:34.674404 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7236860f-e14e-4ff8-a66f-b1cadb6496a1 service cinder] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:16:34.682988 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1030/2075] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:16:34 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:16:44.502260 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=41 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:16:44.679079 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=28 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:16:46.988714 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c5524c-e3ed-4e8c-9f78-e32e1f8f430d service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:16:46.997351 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c5524c-e3ed-4e8c-9f78-e32e1f8f430d service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:16:47.000976 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acf6a45-bf8b-439a-9676-2c72c1b67f69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:16:47.001570 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acf6a45-bf8b-439a-9676-2c72c1b67f69 service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:16:47.001886 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acf6a45-bf8b-439a-9676-2c72c1b67f69 service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:16:47.002161 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acf6a45-bf8b-439a-9676-2c72c1b67f69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:16:47.003877 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-7acf6a45-bf8b-439a-9676-2c72c1b67f69 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:16:47.073851 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acf6a45-bf8b-439a-9676-2c72c1b67f69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:16:47.075208 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acf6a45-bf8b-439a-9676-2c72c1b67f69 service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:16:47.084131 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1046/2076] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:16:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:16:57.080108 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=33 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:17:33.985299 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7236860f-e14e-4ff8-a66f-b1cadb6496a1 service cinder] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:17:33.993737 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7236860f-e14e-4ff8-a66f-b1cadb6496a1 service cinder] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:17:33.997553 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d558f900-bfd7-4bad-94d8-cabbdb1eafbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:17:33.998193 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d558f900-bfd7-4bad-94d8-cabbdb1eafbd service cinder] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:17:33.998477 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d558f900-bfd7-4bad-94d8-cabbdb1eafbd service cinder] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:17:33.998753 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d558f900-bfd7-4bad-94d8-cabbdb1eafbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:17:34.001170 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-d558f900-bfd7-4bad-94d8-cabbdb1eafbd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:17:34.058303 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d558f900-bfd7-4bad-94d8-cabbdb1eafbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:17:34.059729 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d558f900-bfd7-4bad-94d8-cabbdb1eafbd service cinder] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:17:34.068868 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1031/2077] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:17:33 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:17:44.064878 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=28 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:17:50.615449 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acf6a45-bf8b-439a-9676-2c72c1b67f69 service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:17:50.626889 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-7acf6a45-bf8b-439a-9676-2c72c1b67f69 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:17:50.714840 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acf6a45-bf8b-439a-9676-2c72c1b67f69 service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:17:50.718746 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfaeaa4-b0ce-4b18-9ed8-bad53515a242 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71692d05fd324679b1598feb52b3fe6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:17:50.719355 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfaeaa4-b0ce-4b18-9ed8-bad53515a242 service glance] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:17:50.719637 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfaeaa4-b0ce-4b18-9ed8-bad53515a242 service glance] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:17:50.719914 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfaeaa4-b0ce-4b18-9ed8-bad53515a242 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:17:50.722318 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-0dfaeaa4-b0ce-4b18-9ed8-bad53515a242 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:17:50.738450 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-0dfaeaa4-b0ce-4b18-9ed8-bad53515a242 service glance] Could not find project: 41425b7af0c54491be7d15b42790dee0.: keystone.exception.ProjectNotFound: Could not find project: 41425b7af0c54491be7d15b42790dee0. Oct 15 09:17:50.738450 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 09:17:50.738450 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 09:17:50.738450 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 09:17:50.738450 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 09:17:50.738450 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 09:17:50.738450 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 09:17:50.738450 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 09:17:50.738450 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 09:17:50.738450 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 09:17:50.738450 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 09:17:50.738450 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 09:17:50.738450 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Oct 15 09:17:50.738450 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Oct 15 09:17:50.738450 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Oct 15 09:17:50.739501 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Oct 15 09:17:50.739501 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Oct 15 09:17:50.739501 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Oct 15 09:17:50.739501 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 09:17:50.739501 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 09:17:50.739501 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Oct 15 09:17:50.739501 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Oct 15 09:17:50.739501 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 09:17:50.739501 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 09:17:50.739501 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 09:17:50.739501 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 09:17:50.739501 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 09:17:50.739501 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 09:17:50.739501 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 09:17:50.739501 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 09:17:50.739501 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 09:17:50.739501 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 09:17:50.740534 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 09:17:50.740534 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 09:17:50.740534 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 09:17:50.740534 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 09:17:50.740534 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Oct 15 09:17:50.740534 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Oct 15 09:17:50.740534 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Oct 15 09:17:50.740534 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._validate_token_resources() Oct 15 09:17:50.740534 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Oct 15 09:17:50.740534 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Oct 15 09:17:50.740534 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Oct 15 09:17:50.740534 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Oct 15 09:17:50.740534 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 09:17:50.740534 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 09:17:50.740534 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 09:17:50.740534 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 09:17:50.740534 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 09:17:50.741636 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 09:17:50.741636 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 09:17:50.741636 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Oct 15 09:17:50.741636 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 09:17:50.741636 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Oct 15 09:17:50.741636 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 09:17:50.741636 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 09:17:50.741636 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 09:17:50.741636 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Oct 15 09:17:50.741636 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 09:17:50.741636 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Oct 15 09:17:50.741636 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 09:17:50.741636 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 09:17:50.741636 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 09:17:50.741636 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 09:17:50.741636 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 09:17:50.741636 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 09:17:50.742994 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 41425b7af0c54491be7d15b42790dee0. Oct 15 09:17:50.742994 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Oct 15 09:17:50.742994 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1047/2078] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:17:50 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 125 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Oct 15 09:18:00.727455 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=33 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:18:01.881544 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d558f900-bfd7-4bad-94d8-cabbdb1eafbd service cinder] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:01.883044 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-d558f900-bfd7-4bad-94d8-cabbdb1eafbd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:18:01.980743 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d558f900-bfd7-4bad-94d8-cabbdb1eafbd service cinder] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:01.985128 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d634c8-a09d-4067-95ad-6c8426232bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:01.985775 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d634c8-a09d-4067-95ad-6c8426232bcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:01.986057 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d634c8-a09d-4067-95ad-6c8426232bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:01.986331 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d634c8-a09d-4067-95ad-6c8426232bcc admin admin] PATH_INFO: `/v3/users/de5f56dc1d614f11b327170320a64e31` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:01.987481 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d634c8-a09d-4067-95ad-6c8426232bcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=de5f56dc1d614f11b327170320a64e31)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:01.989387 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d634c8-a09d-4067-95ad-6c8426232bcc admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:02.035330 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-84d634c8-a09d-4067-95ad-6c8426232bcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de5f56dc1d614f11b327170320a64e31'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:02.035915 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-84d634c8-a09d-4067-95ad-6c8426232bcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de5f56dc1d614f11b327170320a64e31'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:02.047792 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-84d634c8-a09d-4067-95ad-6c8426232bcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de5f56dc1d614f11b327170320a64e31'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:02.055597 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-84d634c8-a09d-4067-95ad-6c8426232bcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de5f56dc1d614f11b327170320a64e31'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:02.056392 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-84d634c8-a09d-4067-95ad-6c8426232bcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de5f56dc1d614f11b327170320a64e31'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:02.069266 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1032/2079] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 09:18:01 2022] DELETE /identity/v3/users/de5f56dc1d614f11b327170320a64e31 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:18:02.073888 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a962f7f-6830-4fc3-a837-40c72aa0f14d None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:02.074356 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a962f7f-6830-4fc3-a837-40c72aa0f14d None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:02.074452 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a962f7f-6830-4fc3-a837-40c72aa0f14d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:02.102463 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-9a962f7f-6830-4fc3-a837-40c72aa0f14d None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:18:02.103651 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-9a962f7f-6830-4fc3-a837-40c72aa0f14d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:18:02.161495 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1048/2080] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:18:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:18:02.175605 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d634c8-a09d-4067-95ad-6c8426232bcc admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:02.176942 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-84d634c8-a09d-4067-95ad-6c8426232bcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:18:02.235537 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d634c8-a09d-4067-95ad-6c8426232bcc admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:02.239950 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7d5322-f4dd-4c30-801e-8e8e71ba4469 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:02.240546 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7d5322-f4dd-4c30-801e-8e8e71ba4469 service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:02.240823 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7d5322-f4dd-4c30-801e-8e8e71ba4469 service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:02.241100 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7d5322-f4dd-4c30-801e-8e8e71ba4469 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:02.246864 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:18:02.250485 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7d5322-f4dd-4c30-801e-8e8e71ba4469 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:02.252459 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7d5322-f4dd-4c30-801e-8e8e71ba4469 service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:02.263995 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1033/2081] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:18:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:18:02.346771 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a962f7f-6830-4fc3-a837-40c72aa0f14d None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:02.348161 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-9a962f7f-6830-4fc3-a837-40c72aa0f14d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:18:02.360279 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a962f7f-6830-4fc3-a837-40c72aa0f14d None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:02.366019 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6709a4-0b92-407c-a13a-0d9b335006ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:02.366658 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6709a4-0b92-407c-a13a-0d9b335006ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:02.366940 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6709a4-0b92-407c-a13a-0d9b335006ce admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:02.367218 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6709a4-0b92-407c-a13a-0d9b335006ce admin admin] PATH_INFO: `/v3/users/7c1d5294efe4497eb2fe8b6b6982b5fd` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:02.368358 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6709a4-0b92-407c-a13a-0d9b335006ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c1d5294efe4497eb2fe8b6b6982b5fd)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:02.370343 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6709a4-0b92-407c-a13a-0d9b335006ce admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:02.435264 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-fa6709a4-0b92-407c-a13a-0d9b335006ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c1d5294efe4497eb2fe8b6b6982b5fd'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:02.436092 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-fa6709a4-0b92-407c-a13a-0d9b335006ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c1d5294efe4497eb2fe8b6b6982b5fd'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:02.444411 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-fa6709a4-0b92-407c-a13a-0d9b335006ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c1d5294efe4497eb2fe8b6b6982b5fd'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:02.467407 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-fa6709a4-0b92-407c-a13a-0d9b335006ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c1d5294efe4497eb2fe8b6b6982b5fd'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:02.476186 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-fa6709a4-0b92-407c-a13a-0d9b335006ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c1d5294efe4497eb2fe8b6b6982b5fd'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:02.484010 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1049/2082] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 09:18:02 2022] DELETE /identity/v3/users/7c1d5294efe4497eb2fe8b6b6982b5fd => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:18:02.488767 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273fcbe0-cb44-4c61-a9a2-8217d42ee8ae None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:02.489234 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273fcbe0-cb44-4c61-a9a2-8217d42ee8ae None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:02.489573 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273fcbe0-cb44-4c61-a9a2-8217d42ee8ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:02.514517 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-273fcbe0-cb44-4c61-a9a2-8217d42ee8ae None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:18:02.518901 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-273fcbe0-cb44-4c61-a9a2-8217d42ee8ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:18:02.567599 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6709a4-0b92-407c-a13a-0d9b335006ce admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:02.569327 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-fa6709a4-0b92-407c-a13a-0d9b335006ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:18:02.611715 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1034/2083] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:18:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:18:02.620910 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273fcbe0-cb44-4c61-a9a2-8217d42ee8ae None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:02.624689 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-273fcbe0-cb44-4c61-a9a2-8217d42ee8ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:18:02.672815 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6709a4-0b92-407c-a13a-0d9b335006ce admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:02.681397 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6279b7c-8c24-45e7-9881-1656bb4cd482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:02.683951 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6279b7c-8c24-45e7-9881-1656bb4cd482 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:02.684266 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6279b7c-8c24-45e7-9881-1656bb4cd482 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:02.684558 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6279b7c-8c24-45e7-9881-1656bb4cd482 admin admin] PATH_INFO: `/v3/projects/031beaae6e144a7e923be1193b7cb4c8` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:02.686383 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6279b7c-8c24-45e7-9881-1656bb4cd482 admin admin] RBAC: Authorizing `identity:delete_project(project_id=031beaae6e144a7e923be1193b7cb4c8)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:02.688627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6279b7c-8c24-45e7-9881-1656bb4cd482 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:02.699896 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:18:02.738473 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273fcbe0-cb44-4c61-a9a2-8217d42ee8ae None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:02.753061 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-c6279b7c-8c24-45e7-9881-1656bb4cd482 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '031beaae6e144a7e923be1193b7cb4c8', 'request_id': 'req-c6279b7c-8c24-45e7-9881-1656bb4cd482'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:02.761089 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-c6279b7c-8c24-45e7-9881-1656bb4cd482 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '031beaae6e144a7e923be1193b7cb4c8', 'request_id': 'req-c6279b7c-8c24-45e7-9881-1656bb4cd482'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:02.770753 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-c6279b7c-8c24-45e7-9881-1656bb4cd482 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '031beaae6e144a7e923be1193b7cb4c8', 'request_id': 'req-c6279b7c-8c24-45e7-9881-1656bb4cd482'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:02.787213 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-c6279b7c-8c24-45e7-9881-1656bb4cd482 admin admin] The token cache is being invalidate because project 031beaae6e144a7e923be1193b7cb4c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 09:18:02.787807 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-c6279b7c-8c24-45e7-9881-1656bb4cd482 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:02.788601 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-c6279b7c-8c24-45e7-9881-1656bb4cd482 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:02.797863 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1050/2084] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 09:18:02 2022] DELETE /identity/v3/projects/031beaae6e144a7e923be1193b7cb4c8 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 09:18:02.801969 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bf2a34-093e-44e4-9242-0ae4ab27b039 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:02.802937 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bf2a34-093e-44e4-9242-0ae4ab27b039 service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:02.803423 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bf2a34-093e-44e4-9242-0ae4ab27b039 service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:02.804021 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bf2a34-093e-44e4-9242-0ae4ab27b039 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:02.807558 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-d6bf2a34-093e-44e4-9242-0ae4ab27b039 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:18:02.900364 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bf2a34-093e-44e4-9242-0ae4ab27b039 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:02.902022 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bf2a34-093e-44e4-9242-0ae4ab27b039 service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:02.910051 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:18:02.918292 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1035/2085] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:18:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 09:18:03.201175 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6279b7c-8c24-45e7-9881-1656bb4cd482 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:03.202704 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-c6279b7c-8c24-45e7-9881-1656bb4cd482 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:18:03.218898 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6279b7c-8c24-45e7-9881-1656bb4cd482 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:03.223661 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee067129-7ee1-4d0e-bbec-60bdb885809c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:03.225406 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee067129-7ee1-4d0e-bbec-60bdb885809c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:03.225847 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee067129-7ee1-4d0e-bbec-60bdb885809c admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:03.226241 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee067129-7ee1-4d0e-bbec-60bdb885809c admin admin] PATH_INFO: `/v3/projects/b3c3719c7c56456cab90d697730dbaaf` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:03.227564 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee067129-7ee1-4d0e-bbec-60bdb885809c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3c3719c7c56456cab90d697730dbaaf)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:03.229656 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee067129-7ee1-4d0e-bbec-60bdb885809c admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:03.279042 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-ee067129-7ee1-4d0e-bbec-60bdb885809c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3c3719c7c56456cab90d697730dbaaf', 'request_id': 'req-ee067129-7ee1-4d0e-bbec-60bdb885809c'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:03.285654 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-ee067129-7ee1-4d0e-bbec-60bdb885809c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3c3719c7c56456cab90d697730dbaaf', 'request_id': 'req-ee067129-7ee1-4d0e-bbec-60bdb885809c'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:03.293580 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-ee067129-7ee1-4d0e-bbec-60bdb885809c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3c3719c7c56456cab90d697730dbaaf', 'request_id': 'req-ee067129-7ee1-4d0e-bbec-60bdb885809c'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:03.304636 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-ee067129-7ee1-4d0e-bbec-60bdb885809c admin admin] The token cache is being invalidate because project b3c3719c7c56456cab90d697730dbaaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 09:18:03.304927 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-ee067129-7ee1-4d0e-bbec-60bdb885809c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:03.305551 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-ee067129-7ee1-4d0e-bbec-60bdb885809c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:03.307068 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1051/2086] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 09:18:03 2022] DELETE /identity/v3/projects/b3c3719c7c56456cab90d697730dbaaf => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:18:03.311136 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bf2a34-093e-44e4-9242-0ae4ab27b039 service neutron] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:03.321310 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bf2a34-093e-44e4-9242-0ae4ab27b039 service neutron] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:03.380700 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce6f3ee-81b9-414b-b5eb-2ba9063ca222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:03.381340 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6f3ee-81b9-414b-b5eb-2ba9063ca222 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:03.381626 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6f3ee-81b9-414b-b5eb-2ba9063ca222 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:03.381936 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6f3ee-81b9-414b-b5eb-2ba9063ca222 admin admin] PATH_INFO: `/v3/users/1c49a29ddcb84fb3b274fbc0fd8e2928` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:03.383076 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce6f3ee-81b9-414b-b5eb-2ba9063ca222 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c49a29ddcb84fb3b274fbc0fd8e2928)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:03.385125 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce6f3ee-81b9-414b-b5eb-2ba9063ca222 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:03.435224 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-2ce6f3ee-81b9-414b-b5eb-2ba9063ca222 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c49a29ddcb84fb3b274fbc0fd8e2928'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:03.435883 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-2ce6f3ee-81b9-414b-b5eb-2ba9063ca222 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c49a29ddcb84fb3b274fbc0fd8e2928'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:03.447485 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-2ce6f3ee-81b9-414b-b5eb-2ba9063ca222 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c49a29ddcb84fb3b274fbc0fd8e2928'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:03.454508 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-2ce6f3ee-81b9-414b-b5eb-2ba9063ca222 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c49a29ddcb84fb3b274fbc0fd8e2928'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:03.455274 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-2ce6f3ee-81b9-414b-b5eb-2ba9063ca222 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c49a29ddcb84fb3b274fbc0fd8e2928'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:03.467380 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1036/2087] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 09:18:03 2022] DELETE /identity/v3/users/1c49a29ddcb84fb3b274fbc0fd8e2928 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:18:03.736518 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee067129-7ee1-4d0e-bbec-60bdb885809c admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:03.738980 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-ee067129-7ee1-4d0e-bbec-60bdb885809c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:18:03.760365 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee067129-7ee1-4d0e-bbec-60bdb885809c admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:03.826924 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3fdeb1-742b-41a9-97c1-33df9127ef1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:03.827394 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3fdeb1-742b-41a9-97c1-33df9127ef1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:03.827686 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3fdeb1-742b-41a9-97c1-33df9127ef1c admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:03.828087 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3fdeb1-742b-41a9-97c1-33df9127ef1c admin admin] PATH_INFO: `/v3/projects/34a1a128d6b84b1a9529754bc3b2aa49` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:03.829811 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3fdeb1-742b-41a9-97c1-33df9127ef1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=34a1a128d6b84b1a9529754bc3b2aa49)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:03.832704 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3fdeb1-742b-41a9-97c1-33df9127ef1c admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:03.865222 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:18:03.900584 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-6b3fdeb1-742b-41a9-97c1-33df9127ef1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34a1a128d6b84b1a9529754bc3b2aa49', 'request_id': 'req-6b3fdeb1-742b-41a9-97c1-33df9127ef1c'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:03.907633 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-6b3fdeb1-742b-41a9-97c1-33df9127ef1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34a1a128d6b84b1a9529754bc3b2aa49', 'request_id': 'req-6b3fdeb1-742b-41a9-97c1-33df9127ef1c'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:03.917960 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-6b3fdeb1-742b-41a9-97c1-33df9127ef1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34a1a128d6b84b1a9529754bc3b2aa49', 'request_id': 'req-6b3fdeb1-742b-41a9-97c1-33df9127ef1c'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:03.932147 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-6b3fdeb1-742b-41a9-97c1-33df9127ef1c admin admin] The token cache is being invalidate because project 34a1a128d6b84b1a9529754bc3b2aa49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 09:18:03.932577 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-6b3fdeb1-742b-41a9-97c1-33df9127ef1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:03.933358 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-6b3fdeb1-742b-41a9-97c1-33df9127ef1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:03.938229 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1052/2088] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 09:18:03 2022] DELETE /identity/v3/projects/34a1a128d6b84b1a9529754bc3b2aa49 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:18:03.940830 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30e8df3-c2c2-4a1c-8348-195edfb73b3f None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:03.941299 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30e8df3-c2c2-4a1c-8348-195edfb73b3f None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:03.941508 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30e8df3-c2c2-4a1c-8348-195edfb73b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:04.004276 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-c30e8df3-c2c2-4a1c-8348-195edfb73b3f None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:18:04.005576 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-c30e8df3-c2c2-4a1c-8348-195edfb73b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:18:04.058440 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:18:04.072156 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1037/2089] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:18:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 09:18:04.081596 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4328629-fa72-4a92-8ec4-af6b35c26eee None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:04.081923 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4328629-fa72-4a92-8ec4-af6b35c26eee None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:04.082204 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4328629-fa72-4a92-8ec4-af6b35c26eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:04.101905 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-e4328629-fa72-4a92-8ec4-af6b35c26eee None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:18:04.103124 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-e4328629-fa72-4a92-8ec4-af6b35c26eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:18:04.111010 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1053/2090] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:18:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:18:04.115250 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30e8df3-c2c2-4a1c-8348-195edfb73b3f None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:04.124929 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30e8df3-c2c2-4a1c-8348-195edfb73b3f None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:04.129983 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ba77cf-e0e6-464e-ab6e-7ffcb0b4dd4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:04.130626 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba77cf-e0e6-464e-ab6e-7ffcb0b4dd4f admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:04.130902 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba77cf-e0e6-464e-ab6e-7ffcb0b4dd4f admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:04.131172 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba77cf-e0e6-464e-ab6e-7ffcb0b4dd4f admin admin] PATH_INFO: `/v3/domains` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:04.131802 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ba77cf-e0e6-464e-ab6e-7ffcb0b4dd4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 09:18:04.132099 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ba77cf-e0e6-464e-ab6e-7ffcb0b4dd4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:04.133769 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ba77cf-e0e6-464e-ab6e-7ffcb0b4dd4f admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:04.149329 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1038/2091] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 09:18:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 09:18:04.150555 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4328629-fa72-4a92-8ec4-af6b35c26eee None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:04.160468 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4328629-fa72-4a92-8ec4-af6b35c26eee None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:04.165161 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8a5658-b62c-46e2-93a5-9eea0dbf27d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:04.165797 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8a5658-b62c-46e2-93a5-9eea0dbf27d8 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:04.166083 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8a5658-b62c-46e2-93a5-9eea0dbf27d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:04.166359 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8a5658-b62c-46e2-93a5-9eea0dbf27d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:04.167017 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8a5658-b62c-46e2-93a5-9eea0dbf27d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:04.168942 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8a5658-b62c-46e2-93a5-9eea0dbf27d8 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:04.196894 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1054/2092] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 09:18:04 2022] POST /identity/v3/projects => generated 365 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 09:18:04.201060 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ba77cf-e0e6-464e-ab6e-7ffcb0b4dd4f admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:04.211787 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ba77cf-e0e6-464e-ab6e-7ffcb0b4dd4f admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:04.268976 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09656681-8e71-424d-a4ab-ab510d39b1da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:04.269584 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09656681-8e71-424d-a4ab-ab510d39b1da admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:04.269898 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09656681-8e71-424d-a4ab-ab510d39b1da admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:04.270180 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09656681-8e71-424d-a4ab-ab510d39b1da admin admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:04.270823 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09656681-8e71-424d-a4ab-ab510d39b1da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:04.272746 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09656681-8e71-424d-a4ab-ab510d39b1da admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:04.292562 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1039/2093] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 09:18:04 2022] POST /identity/v3/users => generated 283 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:18:04.296862 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8a5658-b62c-46e2-93a5-9eea0dbf27d8 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:04.308481 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8a5658-b62c-46e2-93a5-9eea0dbf27d8 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:04.313048 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef9e956-d568-4359-b903-37cd35239498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:04.313637 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef9e956-d568-4359-b903-37cd35239498 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:04.314030 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef9e956-d568-4359-b903-37cd35239498 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:04.314289 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef9e956-d568-4359-b903-37cd35239498 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:04.314931 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef9e956-d568-4359-b903-37cd35239498 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:04.317103 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef9e956-d568-4359-b903-37cd35239498 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:04.326861 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1055/2094] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:18:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:18:04.330719 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09656681-8e71-424d-a4ab-ab510d39b1da admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:04.340664 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09656681-8e71-424d-a4ab-ab510d39b1da admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:04.345251 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4e277a-239a-4ae9-85de-929fe3189afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:04.345883 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e277a-239a-4ae9-85de-929fe3189afa admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:04.346170 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e277a-239a-4ae9-85de-929fe3189afa admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:04.346475 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e277a-239a-4ae9-85de-929fe3189afa admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:04.347116 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4e277a-239a-4ae9-85de-929fe3189afa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:04.348703 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4e277a-239a-4ae9-85de-929fe3189afa admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:04.359164 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1040/2095] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:18:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:18:04.363529 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef9e956-d568-4359-b903-37cd35239498 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:04.373308 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef9e956-d568-4359-b903-37cd35239498 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:04.378425 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d4cfc0-f165-4ca7-be11-44cbc425948c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:04.379061 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4cfc0-f165-4ca7-be11-44cbc425948c admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:04.379338 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4cfc0-f165-4ca7-be11-44cbc425948c admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:04.379611 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4cfc0-f165-4ca7-be11-44cbc425948c admin admin] PATH_INFO: `/v3/projects/f0c92e72925c4e9c828670fbca5dab42/users/287ee9a523ec45898e2fa4b639215ab9/roles/3248f0268738489b883041851d681bb7` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:04.394114 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d4cfc0-f165-4ca7-be11-44cbc425948c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0c92e72925c4e9c828670fbca5dab42, user_id=287ee9a523ec45898e2fa4b639215ab9, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:04.396744 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d4cfc0-f165-4ca7-be11-44cbc425948c admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:04.409520 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1056/2096] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:18:04 2022] PUT /identity/v3/projects/f0c92e72925c4e9c828670fbca5dab42/users/287ee9a523ec45898e2fa4b639215ab9/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:18:04.419877 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f644f3-ca07-4850-ac67-e472092c0067 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:04.420167 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f644f3-ca07-4850-ac67-e472092c0067 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:04.420738 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f644f3-ca07-4850-ac67-e472092c0067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:04.448783 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-36f644f3-ca07-4850-ac67-e472092c0067 None None] MFA Rules not processed for user `287ee9a523ec45898e2fa4b639215ab9`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:18:04.450055 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-36f644f3-ca07-4850-ac67-e472092c0067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:18:04.511905 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1041/2097] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:18:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:18:04.516443 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d4cfc0-f165-4ca7-be11-44cbc425948c admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:04.577479 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d4cfc0-f165-4ca7-be11-44cbc425948c admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:04.582809 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb9eb8-df58-4c9e-bffa-33e3716fcb8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:04.583409 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb9eb8-df58-4c9e-bffa-33e3716fcb8d admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:04.583692 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb9eb8-df58-4c9e-bffa-33e3716fcb8d admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:04.583971 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb9eb8-df58-4c9e-bffa-33e3716fcb8d admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:04.584595 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cb9eb8-df58-4c9e-bffa-33e3716fcb8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:04.586549 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cb9eb8-df58-4c9e-bffa-33e3716fcb8d admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:04.613303 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1057/2098] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 09:18:04 2022] POST /identity/v3/projects => generated 365 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 09:18:04.616954 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f644f3-ca07-4850-ac67-e472092c0067 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:04.627565 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f644f3-ca07-4850-ac67-e472092c0067 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:04.674311 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:18:04.684536 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978a25ab-1c62-47de-975a-edd5a585f617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:04.685145 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978a25ab-1c62-47de-975a-edd5a585f617 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:04.685433 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978a25ab-1c62-47de-975a-edd5a585f617 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:04.685737 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978a25ab-1c62-47de-975a-edd5a585f617 admin admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:04.686372 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978a25ab-1c62-47de-975a-edd5a585f617 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:04.688305 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978a25ab-1c62-47de-975a-edd5a585f617 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:04.709523 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1042/2099] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 09:18:04 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:18:04.713493 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb9eb8-df58-4c9e-bffa-33e3716fcb8d admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:04.723594 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb9eb8-df58-4c9e-bffa-33e3716fcb8d admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:04.728189 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a029ad04-d0db-4f07-8d04-558a80025173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:04.728781 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a029ad04-d0db-4f07-8d04-558a80025173 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:04.729066 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a029ad04-d0db-4f07-8d04-558a80025173 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:04.729346 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a029ad04-d0db-4f07-8d04-558a80025173 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:04.730026 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a029ad04-d0db-4f07-8d04-558a80025173 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:04.731632 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a029ad04-d0db-4f07-8d04-558a80025173 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:04.741959 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1058/2100] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:18:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:18:04.745707 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978a25ab-1c62-47de-975a-edd5a585f617 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:04.756129 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978a25ab-1c62-47de-975a-edd5a585f617 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:04.760730 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561ff5a9-2fe1-461f-b9b9-2d35084322ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:04.761372 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561ff5a9-2fe1-461f-b9b9-2d35084322ec admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:04.761657 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561ff5a9-2fe1-461f-b9b9-2d35084322ec admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:04.761965 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561ff5a9-2fe1-461f-b9b9-2d35084322ec admin admin] PATH_INFO: `/v3/projects/13848d8905c641f49c15a07e5e1bc794/users/79dfd852f4c34e1f99a5dedd3f6d3b8e/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:04.776894 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561ff5a9-2fe1-461f-b9b9-2d35084322ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=13848d8905c641f49c15a07e5e1bc794, user_id=79dfd852f4c34e1f99a5dedd3f6d3b8e, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:04.779578 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561ff5a9-2fe1-461f-b9b9-2d35084322ec admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:04.792140 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1043/2101] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:18:04 2022] PUT /identity/v3/projects/13848d8905c641f49c15a07e5e1bc794/users/79dfd852f4c34e1f99a5dedd3f6d3b8e/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:18:04.796421 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a029ad04-d0db-4f07-8d04-558a80025173 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:04.806304 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a029ad04-d0db-4f07-8d04-558a80025173 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:04.823120 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:18:04.863230 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2770802-3cfb-44d5-9e86-585261bb0a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:04.863826 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2770802-3cfb-44d5-9e86-585261bb0a86 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:04.864111 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2770802-3cfb-44d5-9e86-585261bb0a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:04.864387 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2770802-3cfb-44d5-9e86-585261bb0a86 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:04.865014 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2770802-3cfb-44d5-9e86-585261bb0a86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:04.866632 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2770802-3cfb-44d5-9e86-585261bb0a86 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:04.876146 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1059/2102] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:18:04 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:18:04.880670 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561ff5a9-2fe1-461f-b9b9-2d35084322ec admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:04.890925 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561ff5a9-2fe1-461f-b9b9-2d35084322ec admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:04.896299 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5515d2b1-98e7-42ac-8418-379d55c26efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:04.896907 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5515d2b1-98e7-42ac-8418-379d55c26efd admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:04.897196 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5515d2b1-98e7-42ac-8418-379d55c26efd admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:04.897473 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5515d2b1-98e7-42ac-8418-379d55c26efd admin admin] PATH_INFO: `/v3/projects/13848d8905c641f49c15a07e5e1bc794/users/79dfd852f4c34e1f99a5dedd3f6d3b8e/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:04.899407 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5515d2b1-98e7-42ac-8418-379d55c26efd admin admin] RBAC: Authorizing `identity:create_grant(project_id=13848d8905c641f49c15a07e5e1bc794, user_id=79dfd852f4c34e1f99a5dedd3f6d3b8e, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:04.902160 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5515d2b1-98e7-42ac-8418-379d55c26efd admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:04.912626 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1044/2103] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:18:04 2022] PUT /identity/v3/projects/13848d8905c641f49c15a07e5e1bc794/users/79dfd852f4c34e1f99a5dedd3f6d3b8e/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:18:04.922969 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed05251a-48fc-4562-8710-3b1901cd4435 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:04.923255 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed05251a-48fc-4562-8710-3b1901cd4435 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:04.923538 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed05251a-48fc-4562-8710-3b1901cd4435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:04.953818 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-ed05251a-48fc-4562-8710-3b1901cd4435 None None] MFA Rules not processed for user `79dfd852f4c34e1f99a5dedd3f6d3b8e`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:18:04.955143 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-ed05251a-48fc-4562-8710-3b1901cd4435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:18:05.025505 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1060/2104] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:18:04 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:18:05.036759 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5515d2b1-98e7-42ac-8418-379d55c26efd admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:05.038543 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-5515d2b1-98e7-42ac-8418-379d55c26efd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:18:05.106100 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5515d2b1-98e7-42ac-8418-379d55c26efd admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:05.110002 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a1aa2c-2364-4c87-b118-3032c69f9621 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:05.110676 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a1aa2c-2364-4c87-b118-3032c69f9621 service cinder] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:05.110957 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a1aa2c-2364-4c87-b118-3032c69f9621 service cinder] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:05.111230 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a1aa2c-2364-4c87-b118-3032c69f9621 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:05.121247 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a1aa2c-2364-4c87-b118-3032c69f9621 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:05.122727 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a1aa2c-2364-4c87-b118-3032c69f9621 service cinder] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:05.134009 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1045/2105] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:18:05 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:18:05.245885 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed05251a-48fc-4562-8710-3b1901cd4435 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:05.255046 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed05251a-48fc-4562-8710-3b1901cd4435 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:05.261976 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f36d8c6-2f50-4835-8d3c-f2b4101d0481 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:05.261976 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36d8c6-2f50-4835-8d3c-f2b4101d0481 service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:05.261976 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36d8c6-2f50-4835-8d3c-f2b4101d0481 service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:05.261976 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36d8c6-2f50-4835-8d3c-f2b4101d0481 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:05.317041 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f36d8c6-2f50-4835-8d3c-f2b4101d0481 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:05.318487 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f36d8c6-2f50-4835-8d3c-f2b4101d0481 service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:05.327252 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1061/2106] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:18:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 09:18:10.651588 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a1aa2c-2364-4c87-b118-3032c69f9621 service cinder] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:10.653645 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-18a1aa2c-2364-4c87-b118-3032c69f9621 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:18:10.719390 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a1aa2c-2364-4c87-b118-3032c69f9621 service cinder] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:10.723834 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d511a55f-5315-4ac3-bdb0-71d5cc0d3efc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:10.724447 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d511a55f-5315-4ac3-bdb0-71d5cc0d3efc service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:10.724731 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d511a55f-5315-4ac3-bdb0-71d5cc0d3efc service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:10.725007 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d511a55f-5315-4ac3-bdb0-71d5cc0d3efc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:10.726766 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-d511a55f-5315-4ac3-bdb0-71d5cc0d3efc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:18:10.736842 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d511a55f-5315-4ac3-bdb0-71d5cc0d3efc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:10.738847 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d511a55f-5315-4ac3-bdb0-71d5cc0d3efc service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:10.747120 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1046/2107] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:18:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:18:12.214019 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f36d8c6-2f50-4835-8d3c-f2b4101d0481 service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:12.256966 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:18:12.278558 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f36d8c6-2f50-4835-8d3c-f2b4101d0481 service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:12.283344 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dadd5b-d506-4787-b2c0-baf99d2cb103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:12.284031 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dadd5b-d506-4787-b2c0-baf99d2cb103 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:12.284367 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dadd5b-d506-4787-b2c0-baf99d2cb103 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:12.284642 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dadd5b-d506-4787-b2c0-baf99d2cb103 admin admin] PATH_INFO: `/v3/users/287ee9a523ec45898e2fa4b639215ab9` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:12.285856 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dadd5b-d506-4787-b2c0-baf99d2cb103 admin admin] RBAC: Authorizing `identity:delete_user(user_id=287ee9a523ec45898e2fa4b639215ab9)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:12.287751 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dadd5b-d506-4787-b2c0-baf99d2cb103 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:12.343877 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-56dadd5b-d506-4787-b2c0-baf99d2cb103 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '287ee9a523ec45898e2fa4b639215ab9'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:12.345261 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-56dadd5b-d506-4787-b2c0-baf99d2cb103 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '287ee9a523ec45898e2fa4b639215ab9'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:12.371467 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-56dadd5b-d506-4787-b2c0-baf99d2cb103 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '287ee9a523ec45898e2fa4b639215ab9'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:12.393948 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-56dadd5b-d506-4787-b2c0-baf99d2cb103 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '287ee9a523ec45898e2fa4b639215ab9'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:12.408312 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-56dadd5b-d506-4787-b2c0-baf99d2cb103 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '287ee9a523ec45898e2fa4b639215ab9'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:12.415962 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1062/2108] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 09:18:12 2022] DELETE /identity/v3/users/287ee9a523ec45898e2fa4b639215ab9 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:18:12.420309 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22ca6b6-bcc6-4ca2-ab66-0569adbcbe7e None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:12.421021 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22ca6b6-bcc6-4ca2-ab66-0569adbcbe7e None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:12.421139 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22ca6b6-bcc6-4ca2-ab66-0569adbcbe7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:12.444659 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-f22ca6b6-bcc6-4ca2-ab66-0569adbcbe7e None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:18:12.445917 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-f22ca6b6-bcc6-4ca2-ab66-0569adbcbe7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:18:12.491178 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:18:12.507762 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1047/2109] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:18:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:18:12.519391 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dadd5b-d506-4787-b2c0-baf99d2cb103 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:12.522184 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-56dadd5b-d506-4787-b2c0-baf99d2cb103 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:18:12.593353 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dadd5b-d506-4787-b2c0-baf99d2cb103 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:12.597716 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179abd54-9377-4d15-b76a-92b5dd8a7ef1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:12.598329 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179abd54-9377-4d15-b76a-92b5dd8a7ef1 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:12.598613 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179abd54-9377-4d15-b76a-92b5dd8a7ef1 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:12.598921 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179abd54-9377-4d15-b76a-92b5dd8a7ef1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:12.608429 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179abd54-9377-4d15-b76a-92b5dd8a7ef1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:12.609883 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179abd54-9377-4d15-b76a-92b5dd8a7ef1 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:12.621999 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1063/2110] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:18:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 09:18:12.893742 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22ca6b6-bcc6-4ca2-ab66-0569adbcbe7e None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:12.895297 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-f22ca6b6-bcc6-4ca2-ab66-0569adbcbe7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:18:12.907674 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22ca6b6-bcc6-4ca2-ab66-0569adbcbe7e None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:12.913175 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91800d4-cdf1-4277-b145-ea9a2415c5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:12.913849 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91800d4-cdf1-4277-b145-ea9a2415c5c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:12.914132 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91800d4-cdf1-4277-b145-ea9a2415c5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:12.914561 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91800d4-cdf1-4277-b145-ea9a2415c5c8 admin admin] PATH_INFO: `/v3/projects/f0c92e72925c4e9c828670fbca5dab42` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:12.915694 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91800d4-cdf1-4277-b145-ea9a2415c5c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0c92e72925c4e9c828670fbca5dab42)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:12.917729 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91800d4-cdf1-4277-b145-ea9a2415c5c8 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:12.966584 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-d91800d4-cdf1-4277-b145-ea9a2415c5c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0c92e72925c4e9c828670fbca5dab42', 'request_id': 'req-d91800d4-cdf1-4277-b145-ea9a2415c5c8'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:12.972946 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-d91800d4-cdf1-4277-b145-ea9a2415c5c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0c92e72925c4e9c828670fbca5dab42', 'request_id': 'req-d91800d4-cdf1-4277-b145-ea9a2415c5c8'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:12.979934 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-d91800d4-cdf1-4277-b145-ea9a2415c5c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0c92e72925c4e9c828670fbca5dab42', 'request_id': 'req-d91800d4-cdf1-4277-b145-ea9a2415c5c8'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:12.990297 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-d91800d4-cdf1-4277-b145-ea9a2415c5c8 admin admin] The token cache is being invalidate because project f0c92e72925c4e9c828670fbca5dab42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 09:18:12.990589 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-d91800d4-cdf1-4277-b145-ea9a2415c5c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:12.991222 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-d91800d4-cdf1-4277-b145-ea9a2415c5c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:12.992776 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1048/2111] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 09:18:12 2022] DELETE /identity/v3/projects/f0c92e72925c4e9c828670fbca5dab42 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:18:12.996881 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179abd54-9377-4d15-b76a-92b5dd8a7ef1 service neutron] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:13.007583 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179abd54-9377-4d15-b76a-92b5dd8a7ef1 service neutron] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:13.065641 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53beb2ae-5352-4ff8-9514-5f98d8b07b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:13.066301 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53beb2ae-5352-4ff8-9514-5f98d8b07b71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:13.066586 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53beb2ae-5352-4ff8-9514-5f98d8b07b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:13.066867 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53beb2ae-5352-4ff8-9514-5f98d8b07b71 admin admin] PATH_INFO: `/v3/users/79dfd852f4c34e1f99a5dedd3f6d3b8e` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:13.067991 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53beb2ae-5352-4ff8-9514-5f98d8b07b71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79dfd852f4c34e1f99a5dedd3f6d3b8e)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:13.069938 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53beb2ae-5352-4ff8-9514-5f98d8b07b71 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:13.072202 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:18:13.117006 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-53beb2ae-5352-4ff8-9514-5f98d8b07b71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79dfd852f4c34e1f99a5dedd3f6d3b8e'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:13.117601 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-53beb2ae-5352-4ff8-9514-5f98d8b07b71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79dfd852f4c34e1f99a5dedd3f6d3b8e'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:13.124523 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-53beb2ae-5352-4ff8-9514-5f98d8b07b71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79dfd852f4c34e1f99a5dedd3f6d3b8e'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:13.137581 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-53beb2ae-5352-4ff8-9514-5f98d8b07b71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79dfd852f4c34e1f99a5dedd3f6d3b8e'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:13.145760 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-53beb2ae-5352-4ff8-9514-5f98d8b07b71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79dfd852f4c34e1f99a5dedd3f6d3b8e'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:13.151829 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1064/2112] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 09:18:12 2022] DELETE /identity/v3/users/79dfd852f4c34e1f99a5dedd3f6d3b8e => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 09:18:13.443998 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91800d4-cdf1-4277-b145-ea9a2415c5c8 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:13.446383 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-d91800d4-cdf1-4277-b145-ea9a2415c5c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:18:13.472813 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91800d4-cdf1-4277-b145-ea9a2415c5c8 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:13.530768 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2437deb-eb46-4398-937f-ca98e568667f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:13.531421 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2437deb-eb46-4398-937f-ca98e568667f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:13.531698 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2437deb-eb46-4398-937f-ca98e568667f admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:13.531970 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2437deb-eb46-4398-937f-ca98e568667f admin admin] PATH_INFO: `/v3/projects/13848d8905c641f49c15a07e5e1bc794` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:13.533093 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2437deb-eb46-4398-937f-ca98e568667f admin admin] RBAC: Authorizing `identity:delete_project(project_id=13848d8905c641f49c15a07e5e1bc794)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:13.535065 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2437deb-eb46-4398-937f-ca98e568667f admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:13.580586 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-e2437deb-eb46-4398-937f-ca98e568667f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13848d8905c641f49c15a07e5e1bc794', 'request_id': 'req-e2437deb-eb46-4398-937f-ca98e568667f'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:13.587038 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-e2437deb-eb46-4398-937f-ca98e568667f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13848d8905c641f49c15a07e5e1bc794', 'request_id': 'req-e2437deb-eb46-4398-937f-ca98e568667f'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:13.594063 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-e2437deb-eb46-4398-937f-ca98e568667f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13848d8905c641f49c15a07e5e1bc794', 'request_id': 'req-e2437deb-eb46-4398-937f-ca98e568667f'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:13.604458 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-e2437deb-eb46-4398-937f-ca98e568667f admin admin] The token cache is being invalidate because project 13848d8905c641f49c15a07e5e1bc794 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 09:18:13.604755 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-e2437deb-eb46-4398-937f-ca98e568667f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:13.605372 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-e2437deb-eb46-4398-937f-ca98e568667f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:13.606863 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1049/2113] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 09:18:13 2022] DELETE /identity/v3/projects/13848d8905c641f49c15a07e5e1bc794 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:18:13.613483 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdea7a6-9575-4feb-bc81-2e2837345662 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:13.613997 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdea7a6-9575-4feb-bc81-2e2837345662 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:13.614094 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdea7a6-9575-4feb-bc81-2e2837345662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:13.633918 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-0fdea7a6-9575-4feb-bc81-2e2837345662 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:18:13.635108 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-0fdea7a6-9575-4feb-bc81-2e2837345662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:18:13.696273 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1065/2114] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:18:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:18:13.704044 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60087d0-fa79-4cfc-a2df-7094bbab6b80 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:13.704395 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60087d0-fa79-4cfc-a2df-7094bbab6b80 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:13.704853 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60087d0-fa79-4cfc-a2df-7094bbab6b80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:13.712915 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:18:13.727103 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-d60087d0-fa79-4cfc-a2df-7094bbab6b80 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:18:13.728976 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-d60087d0-fa79-4cfc-a2df-7094bbab6b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:18:13.737932 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1050/2115] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:18:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:18:13.742297 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdea7a6-9575-4feb-bc81-2e2837345662 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:13.751521 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdea7a6-9575-4feb-bc81-2e2837345662 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:13.756465 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f537b-da06-41f6-a742-0a991bef3c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:13.757325 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f537b-da06-41f6-a742-0a991bef3c52 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:13.757670 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f537b-da06-41f6-a742-0a991bef3c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:13.757979 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f537b-da06-41f6-a742-0a991bef3c52 admin admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:13.758633 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91f537b-da06-41f6-a742-0a991bef3c52 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 09:18:13.758939 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91f537b-da06-41f6-a742-0a991bef3c52 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:13.760554 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91f537b-da06-41f6-a742-0a991bef3c52 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:13.772610 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1066/2116] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 09:18:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 09:18:13.776711 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60087d0-fa79-4cfc-a2df-7094bbab6b80 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:13.786491 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60087d0-fa79-4cfc-a2df-7094bbab6b80 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:13.791437 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41cce01-3aec-4208-8c34-b9621e320047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:13.792050 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41cce01-3aec-4208-8c34-b9621e320047 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:13.792327 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41cce01-3aec-4208-8c34-b9621e320047 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:13.792602 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41cce01-3aec-4208-8c34-b9621e320047 admin admin] PATH_INFO: `/v3/projects` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:13.793241 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41cce01-3aec-4208-8c34-b9621e320047 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:13.795201 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41cce01-3aec-4208-8c34-b9621e320047 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:13.826153 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1051/2117] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 09:18:13 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 09:18:13.827286 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f537b-da06-41f6-a742-0a991bef3c52 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:13.843292 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f537b-da06-41f6-a742-0a991bef3c52 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:13.901853 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a38aef1-1fa4-4796-9d7b-77137671d0cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:13.902387 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a38aef1-1fa4-4796-9d7b-77137671d0cf admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:13.902849 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a38aef1-1fa4-4796-9d7b-77137671d0cf admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:13.903142 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a38aef1-1fa4-4796-9d7b-77137671d0cf admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:13.903807 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a38aef1-1fa4-4796-9d7b-77137671d0cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:13.905720 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a38aef1-1fa4-4796-9d7b-77137671d0cf admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:13.923719 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:18:13.934179 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1067/2118] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 09:18:13 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 09:18:13.935576 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41cce01-3aec-4208-8c34-b9621e320047 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:13.946308 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41cce01-3aec-4208-8c34-b9621e320047 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:13.951242 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafc6d08-fa25-4d73-b132-afa094fcc458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:13.951858 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafc6d08-fa25-4d73-b132-afa094fcc458 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:13.952140 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafc6d08-fa25-4d73-b132-afa094fcc458 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:13.952412 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafc6d08-fa25-4d73-b132-afa094fcc458 admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:13.953054 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafc6d08-fa25-4d73-b132-afa094fcc458 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:13.954683 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafc6d08-fa25-4d73-b132-afa094fcc458 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:13.967841 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1052/2119] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:18:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:18:13.969083 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a38aef1-1fa4-4796-9d7b-77137671d0cf admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:13.979599 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a38aef1-1fa4-4796-9d7b-77137671d0cf admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:13.984208 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc81082a-d999-44e9-83b8-6eff17a5c691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:13.984836 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81082a-d999-44e9-83b8-6eff17a5c691 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:13.985176 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81082a-d999-44e9-83b8-6eff17a5c691 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:13.985457 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81082a-d999-44e9-83b8-6eff17a5c691 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:13.986141 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc81082a-d999-44e9-83b8-6eff17a5c691 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:13.988316 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc81082a-d999-44e9-83b8-6eff17a5c691 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:13.999264 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1068/2120] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:18:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:18:14.002623 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafc6d08-fa25-4d73-b132-afa094fcc458 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:14.012758 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafc6d08-fa25-4d73-b132-afa094fcc458 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:14.017448 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84e8924-6314-4031-af98-3465defe8a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:14.018769 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e8924-6314-4031-af98-3465defe8a86 admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:14.019056 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e8924-6314-4031-af98-3465defe8a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:14.019333 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e8924-6314-4031-af98-3465defe8a86 admin admin] PATH_INFO: `/v3/projects/ba76940a547f41069e3d99b4a07bb505/users/280bb56a4a8c41febb86c00dba513673/roles/3248f0268738489b883041851d681bb7` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:14.034634 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84e8924-6314-4031-af98-3465defe8a86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba76940a547f41069e3d99b4a07bb505, user_id=280bb56a4a8c41febb86c00dba513673, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:14.037305 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84e8924-6314-4031-af98-3465defe8a86 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:14.050913 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1053/2121] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:18:14 2022] PUT /identity/v3/projects/ba76940a547f41069e3d99b4a07bb505/users/280bb56a4a8c41febb86c00dba513673/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 09:18:14.061226 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab627f-32e9-42d7-8c7f-6934234859ce None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:14.061512 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab627f-32e9-42d7-8c7f-6934234859ce None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:14.061849 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab627f-32e9-42d7-8c7f-6934234859ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:14.090478 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-31ab627f-32e9-42d7-8c7f-6934234859ce None None] MFA Rules not processed for user `280bb56a4a8c41febb86c00dba513673`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:18:14.091695 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-31ab627f-32e9-42d7-8c7f-6934234859ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:18:14.160406 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1069/2122] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:18:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:18:14.172222 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84e8924-6314-4031-af98-3465defe8a86 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:14.174016 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-b84e8924-6314-4031-af98-3465defe8a86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:18:14.232238 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84e8924-6314-4031-af98-3465defe8a86 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:14.238114 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7104678-c022-4b8e-b55e-502a7a7a1338 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:14.238724 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7104678-c022-4b8e-b55e-502a7a7a1338 service cinder] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:14.239002 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7104678-c022-4b8e-b55e-502a7a7a1338 service cinder] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:14.239273 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7104678-c022-4b8e-b55e-502a7a7a1338 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:14.247778 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7104678-c022-4b8e-b55e-502a7a7a1338 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:14.249206 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7104678-c022-4b8e-b55e-502a7a7a1338 service cinder] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:14.258667 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1054/2123] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:18:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:18:14.283677 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ab627f-32e9-42d7-8c7f-6934234859ce None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:14.350230 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ab627f-32e9-42d7-8c7f-6934234859ce None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:14.355674 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08a5673-33eb-41a0-8605-01eca8b51cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:14.356329 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08a5673-33eb-41a0-8605-01eca8b51cf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:14.356696 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08a5673-33eb-41a0-8605-01eca8b51cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:14.356955 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08a5673-33eb-41a0-8605-01eca8b51cf1 admin admin] PATH_INFO: `/v3/users/280bb56a4a8c41febb86c00dba513673` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:14.358125 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08a5673-33eb-41a0-8605-01eca8b51cf1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=280bb56a4a8c41febb86c00dba513673)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:14.360009 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08a5673-33eb-41a0-8605-01eca8b51cf1 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:14.419168 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-c08a5673-33eb-41a0-8605-01eca8b51cf1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '280bb56a4a8c41febb86c00dba513673'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:14.419771 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-c08a5673-33eb-41a0-8605-01eca8b51cf1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '280bb56a4a8c41febb86c00dba513673'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:14.428421 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-c08a5673-33eb-41a0-8605-01eca8b51cf1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '280bb56a4a8c41febb86c00dba513673'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:14.442468 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-c08a5673-33eb-41a0-8605-01eca8b51cf1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '280bb56a4a8c41febb86c00dba513673'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:14.446039 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:18:14.450465 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-c08a5673-33eb-41a0-8605-01eca8b51cf1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '280bb56a4a8c41febb86c00dba513673'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:14.458521 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1070/2124] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 09:18:14 2022] DELETE /identity/v3/users/280bb56a4a8c41febb86c00dba513673 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:18:14.463046 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd616439-e7ac-4caf-a53b-40074e74722a None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:14.463516 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd616439-e7ac-4caf-a53b-40074e74722a None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:14.463646 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd616439-e7ac-4caf-a53b-40074e74722a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:14.482569 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-dd616439-e7ac-4caf-a53b-40074e74722a None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:18:14.483762 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-dd616439-e7ac-4caf-a53b-40074e74722a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:18:14.554499 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1055/2125] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:18:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:18:14.564180 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08a5673-33eb-41a0-8605-01eca8b51cf1 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:14.566196 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-c08a5673-33eb-41a0-8605-01eca8b51cf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:18:14.625181 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08a5673-33eb-41a0-8605-01eca8b51cf1 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:14.629421 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbbb38a-3edc-4fed-b29f-b166cd5467e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:14.630084 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbbb38a-3edc-4fed-b29f-b166cd5467e0 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:14.630350 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbbb38a-3edc-4fed-b29f-b166cd5467e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:14.630705 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbbb38a-3edc-4fed-b29f-b166cd5467e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:14.639785 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbbb38a-3edc-4fed-b29f-b166cd5467e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:14.641214 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbbb38a-3edc-4fed-b29f-b166cd5467e0 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:14.651909 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1071/2126] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:18:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:18:14.905791 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd616439-e7ac-4caf-a53b-40074e74722a None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:14.907237 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-dd616439-e7ac-4caf-a53b-40074e74722a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:18:14.919878 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd616439-e7ac-4caf-a53b-40074e74722a None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:14.925853 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1309beb-b10f-451c-840c-2a05515c5933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:14.926503 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1309beb-b10f-451c-840c-2a05515c5933 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:14.926789 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1309beb-b10f-451c-840c-2a05515c5933 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:14.927066 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1309beb-b10f-451c-840c-2a05515c5933 admin admin] PATH_INFO: `/v3/projects/ba76940a547f41069e3d99b4a07bb505` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:14.934719 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1309beb-b10f-451c-840c-2a05515c5933 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba76940a547f41069e3d99b4a07bb505)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:14.936671 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1309beb-b10f-451c-840c-2a05515c5933 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:14.976798 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:18:14.988206 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-c1309beb-b10f-451c-840c-2a05515c5933 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba76940a547f41069e3d99b4a07bb505', 'request_id': 'req-c1309beb-b10f-451c-840c-2a05515c5933'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:14.994747 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-c1309beb-b10f-451c-840c-2a05515c5933 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba76940a547f41069e3d99b4a07bb505', 'request_id': 'req-c1309beb-b10f-451c-840c-2a05515c5933'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:15.004188 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-c1309beb-b10f-451c-840c-2a05515c5933 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba76940a547f41069e3d99b4a07bb505', 'request_id': 'req-c1309beb-b10f-451c-840c-2a05515c5933'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:15.016325 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-c1309beb-b10f-451c-840c-2a05515c5933 admin admin] The token cache is being invalidate because project ba76940a547f41069e3d99b4a07bb505 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 09:18:15.016615 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-c1309beb-b10f-451c-840c-2a05515c5933 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:15.017248 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-c1309beb-b10f-451c-840c-2a05515c5933 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:15.019031 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1056/2127] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 09:18:14 2022] DELETE /identity/v3/projects/ba76940a547f41069e3d99b4a07bb505 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 09:18:15.023809 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1938f28-c827-43bd-bf59-50a3a6ecf699 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:15.024151 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1938f28-c827-43bd-bf59-50a3a6ecf699 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:15.024436 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1938f28-c827-43bd-bf59-50a3a6ecf699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:15.044169 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-e1938f28-c827-43bd-bf59-50a3a6ecf699 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:18:15.045388 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-e1938f28-c827-43bd-bf59-50a3a6ecf699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:18:15.108088 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1072/2128] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:18:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 09:18:15.117370 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32398a23-53c9-481c-a022-3b3e0776cd09 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:15.117662 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32398a23-53c9-481c-a022-3b3e0776cd09 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:15.117978 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32398a23-53c9-481c-a022-3b3e0776cd09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:15.138990 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-32398a23-53c9-481c-a022-3b3e0776cd09 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:18:15.140216 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-32398a23-53c9-481c-a022-3b3e0776cd09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:18:15.148217 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1057/2129] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:18:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:18:15.153550 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1938f28-c827-43bd-bf59-50a3a6ecf699 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:15.163719 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1938f28-c827-43bd-bf59-50a3a6ecf699 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:15.168491 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e268e78-b230-4ef6-9878-326cfaa51c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:15.169191 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e268e78-b230-4ef6-9878-326cfaa51c4f admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:15.169536 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e268e78-b230-4ef6-9878-326cfaa51c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:15.169914 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e268e78-b230-4ef6-9878-326cfaa51c4f admin admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:15.170883 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e268e78-b230-4ef6-9878-326cfaa51c4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 09:18:15.171257 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e268e78-b230-4ef6-9878-326cfaa51c4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:15.172922 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e268e78-b230-4ef6-9878-326cfaa51c4f admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:15.185779 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1073/2130] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 09:18:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 09:18:15.189942 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32398a23-53c9-481c-a022-3b3e0776cd09 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:15.199310 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32398a23-53c9-481c-a022-3b3e0776cd09 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:15.203994 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678415c6-68cf-4de6-90b6-0377ada45bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:15.204589 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678415c6-68cf-4de6-90b6-0377ada45bb5 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:15.204861 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678415c6-68cf-4de6-90b6-0377ada45bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:15.205182 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678415c6-68cf-4de6-90b6-0377ada45bb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:15.205854 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678415c6-68cf-4de6-90b6-0377ada45bb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:15.207833 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678415c6-68cf-4de6-90b6-0377ada45bb5 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:15.238138 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1058/2131] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 09:18:15 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:18:15.239226 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e268e78-b230-4ef6-9878-326cfaa51c4f admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:15.248982 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e268e78-b230-4ef6-9878-326cfaa51c4f admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:15.306639 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2088ca-7942-482a-8e52-b58b0ab73088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:15.307697 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2088ca-7942-482a-8e52-b58b0ab73088 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:15.307989 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2088ca-7942-482a-8e52-b58b0ab73088 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:15.308293 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2088ca-7942-482a-8e52-b58b0ab73088 admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:15.309140 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2088ca-7942-482a-8e52-b58b0ab73088 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:15.311241 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2088ca-7942-482a-8e52-b58b0ab73088 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:15.340181 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1074/2132] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 09:18:15 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:18:15.341347 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678415c6-68cf-4de6-90b6-0377ada45bb5 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:15.356012 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678415c6-68cf-4de6-90b6-0377ada45bb5 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:15.361376 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3b5075-e1cf-4972-b347-6d17753150eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:15.362121 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3b5075-e1cf-4972-b347-6d17753150eb admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:15.363336 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3b5075-e1cf-4972-b347-6d17753150eb admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:15.363449 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3b5075-e1cf-4972-b347-6d17753150eb admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:15.364189 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3b5075-e1cf-4972-b347-6d17753150eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:15.366069 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3b5075-e1cf-4972-b347-6d17753150eb admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:15.380338 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1059/2133] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:18:15 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:18:15.382108 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2088ca-7942-482a-8e52-b58b0ab73088 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:15.392583 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2088ca-7942-482a-8e52-b58b0ab73088 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:15.398358 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0c9e5e-3a69-46da-9cfa-60837918c915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:15.399116 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0c9e5e-3a69-46da-9cfa-60837918c915 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:15.399314 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0c9e5e-3a69-46da-9cfa-60837918c915 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:15.399662 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0c9e5e-3a69-46da-9cfa-60837918c915 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:15.400442 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0c9e5e-3a69-46da-9cfa-60837918c915 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:15.402396 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0c9e5e-3a69-46da-9cfa-60837918c915 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:15.414522 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1075/2134] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:18:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:18:15.418658 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3b5075-e1cf-4972-b347-6d17753150eb admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:15.430133 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3b5075-e1cf-4972-b347-6d17753150eb admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:15.435101 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f73a0b3-f6f3-4660-bc5c-dd363b287a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:15.435764 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73a0b3-f6f3-4660-bc5c-dd363b287a08 admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:15.436045 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73a0b3-f6f3-4660-bc5c-dd363b287a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:15.436456 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73a0b3-f6f3-4660-bc5c-dd363b287a08 admin admin] PATH_INFO: `/v3/projects/9c5258b2dbd7421da1dce3db05030f54/users/db9d9cc63fec4dac87214047576ea2f8/roles/3248f0268738489b883041851d681bb7` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:15.451650 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f73a0b3-f6f3-4660-bc5c-dd363b287a08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c5258b2dbd7421da1dce3db05030f54, user_id=db9d9cc63fec4dac87214047576ea2f8, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:15.454362 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f73a0b3-f6f3-4660-bc5c-dd363b287a08 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:15.478459 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1060/2135] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:18:15 2022] PUT /identity/v3/projects/9c5258b2dbd7421da1dce3db05030f54/users/db9d9cc63fec4dac87214047576ea2f8/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:18:15.489249 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45696c92-5564-4503-b34e-94e95d74b88a None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:15.489606 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45696c92-5564-4503-b34e-94e95d74b88a None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:15.489899 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45696c92-5564-4503-b34e-94e95d74b88a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:15.499551 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:18:15.523064 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-45696c92-5564-4503-b34e-94e95d74b88a None None] MFA Rules not processed for user `db9d9cc63fec4dac87214047576ea2f8`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:18:15.524641 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-45696c92-5564-4503-b34e-94e95d74b88a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:18:15.652769 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1076/2136] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:18:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:18:15.673845 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f73a0b3-f6f3-4660-bc5c-dd363b287a08 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:15.675647 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-1f73a0b3-f6f3-4660-bc5c-dd363b287a08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:18:15.735919 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f73a0b3-f6f3-4660-bc5c-dd363b287a08 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:15.739732 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac19e88-edff-4ecb-89a8-b773e92135cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:15.740342 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac19e88-edff-4ecb-89a8-b773e92135cd service cinder] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:15.740625 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac19e88-edff-4ecb-89a8-b773e92135cd service cinder] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:15.740900 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac19e88-edff-4ecb-89a8-b773e92135cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:15.749823 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac19e88-edff-4ecb-89a8-b773e92135cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:15.751208 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac19e88-edff-4ecb-89a8-b773e92135cd service cinder] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:15.760185 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1061/2137] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:18:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:18:15.851174 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45696c92-5564-4503-b34e-94e95d74b88a None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:15.852630 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-45696c92-5564-4503-b34e-94e95d74b88a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:18:15.974361 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45696c92-5564-4503-b34e-94e95d74b88a None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:15.981164 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c643a6a-fdd0-407a-b101-c9b584354470 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0a8af5cadc4cc1ab73e8eb2a20bb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:15.981823 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c643a6a-fdd0-407a-b101-c9b584354470 service nova] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:15.982110 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c643a6a-fdd0-407a-b101-c9b584354470 service nova] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:15.982388 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c643a6a-fdd0-407a-b101-c9b584354470 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:15.984385 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-2c643a6a-fdd0-407a-b101-c9b584354470 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:18:16.019636 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c643a6a-fdd0-407a-b101-c9b584354470 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:16.021488 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c643a6a-fdd0-407a-b101-c9b584354470 service nova] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:16.041824 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1077/2138] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:18:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:18:17.247927 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d060a-c57f-4df2-b665-d8a53a0eb45a None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:17.248401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d060a-c57f-4df2-b665-d8a53a0eb45a None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:17.248493 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d060a-c57f-4df2-b665-d8a53a0eb45a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:17.267243 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-980d060a-c57f-4df2-b665-d8a53a0eb45a None None] MFA Rules not processed for user `db9d9cc63fec4dac87214047576ea2f8`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:18:17.268447 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-980d060a-c57f-4df2-b665-d8a53a0eb45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:18:17.275971 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1062/2139] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:18:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:18:17.753902 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c643a6a-fdd0-407a-b101-c9b584354470 service nova] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:17.755362 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-2c643a6a-fdd0-407a-b101-c9b584354470 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:18:17.852375 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c643a6a-fdd0-407a-b101-c9b584354470 service nova] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:17.859321 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5c0913-1289-4b77-b58c-fa573b5cae69 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38986ca2612c4fc59d960cd693834f9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:17.860073 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5c0913-1289-4b77-b58c-fa573b5cae69 service placement] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:17.860406 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5c0913-1289-4b77-b58c-fa573b5cae69 service placement] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:17.860755 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5c0913-1289-4b77-b58c-fa573b5cae69 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:17.862860 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-2e5c0913-1289-4b77-b58c-fa573b5cae69 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:18:17.875798 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5c0913-1289-4b77-b58c-fa573b5cae69 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:17.877372 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5c0913-1289-4b77-b58c-fa573b5cae69 service placement] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:17.887746 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1078/2140] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:18:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:18:22.314711 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d060a-c57f-4df2-b665-d8a53a0eb45a None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:22.316272 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-980d060a-c57f-4df2-b665-d8a53a0eb45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:18:22.372136 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d060a-c57f-4df2-b665-d8a53a0eb45a None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:22.375565 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f25444-1d22-403c-b609-20029a965dae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:22.376156 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f25444-1d22-403c-b609-20029a965dae service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:22.376434 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f25444-1d22-403c-b609-20029a965dae service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:22.376711 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f25444-1d22-403c-b609-20029a965dae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:22.378440 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-57f25444-1d22-403c-b609-20029a965dae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:18:22.388245 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:18:22.442021 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f25444-1d22-403c-b609-20029a965dae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:22.443473 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f25444-1d22-403c-b609-20029a965dae service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:22.453699 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1063/2141] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:18:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:18:23.041272 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5c0913-1289-4b77-b58c-fa573b5cae69 service placement] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:23.043230 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-2e5c0913-1289-4b77-b58c-fa573b5cae69 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:18:23.060729 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5c0913-1289-4b77-b58c-fa573b5cae69 service placement] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:23.066270 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad11979-0fe7-4824-b4d8-067be15da062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:23.066908 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad11979-0fe7-4824-b4d8-067be15da062 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:23.067187 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad11979-0fe7-4824-b4d8-067be15da062 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:23.068027 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad11979-0fe7-4824-b4d8-067be15da062 admin admin] PATH_INFO: `/v3/users/d716aed1f7684cda9b82ba0d5c3d24d4` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:23.068619 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad11979-0fe7-4824-b4d8-067be15da062 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d716aed1f7684cda9b82ba0d5c3d24d4)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:23.070548 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad11979-0fe7-4824-b4d8-067be15da062 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:23.073009 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:18:23.117101 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7ad11979-0fe7-4824-b4d8-067be15da062 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd716aed1f7684cda9b82ba0d5c3d24d4'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:23.117805 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7ad11979-0fe7-4824-b4d8-067be15da062 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd716aed1f7684cda9b82ba0d5c3d24d4'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:23.124459 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7ad11979-0fe7-4824-b4d8-067be15da062 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd716aed1f7684cda9b82ba0d5c3d24d4'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:23.136909 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7ad11979-0fe7-4824-b4d8-067be15da062 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd716aed1f7684cda9b82ba0d5c3d24d4'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:23.143870 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7ad11979-0fe7-4824-b4d8-067be15da062 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd716aed1f7684cda9b82ba0d5c3d24d4'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:23.149936 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1079/2142] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 09:18:23 2022] DELETE /identity/v3/users/d716aed1f7684cda9b82ba0d5c3d24d4 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:18:23.324703 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f25444-1d22-403c-b609-20029a965dae service neutron] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:23.337703 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f25444-1d22-403c-b609-20029a965dae service neutron] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:23.395163 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457fae08-b3fd-4168-a4eb-f9e640da20b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:23.395799 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457fae08-b3fd-4168-a4eb-f9e640da20b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:23.396083 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457fae08-b3fd-4168-a4eb-f9e640da20b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:23.396364 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457fae08-b3fd-4168-a4eb-f9e640da20b0 admin admin] PATH_INFO: `/v3/projects/6c350a7b4ae944c188fc8af5a91438f3` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:23.397719 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457fae08-b3fd-4168-a4eb-f9e640da20b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c350a7b4ae944c188fc8af5a91438f3)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:23.399722 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457fae08-b3fd-4168-a4eb-f9e640da20b0 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:23.446500 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-457fae08-b3fd-4168-a4eb-f9e640da20b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c350a7b4ae944c188fc8af5a91438f3', 'request_id': 'req-457fae08-b3fd-4168-a4eb-f9e640da20b0'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:23.452634 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-457fae08-b3fd-4168-a4eb-f9e640da20b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c350a7b4ae944c188fc8af5a91438f3', 'request_id': 'req-457fae08-b3fd-4168-a4eb-f9e640da20b0'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:23.460860 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-457fae08-b3fd-4168-a4eb-f9e640da20b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c350a7b4ae944c188fc8af5a91438f3', 'request_id': 'req-457fae08-b3fd-4168-a4eb-f9e640da20b0'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:23.471061 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-457fae08-b3fd-4168-a4eb-f9e640da20b0 admin admin] The token cache is being invalidate because project 6c350a7b4ae944c188fc8af5a91438f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 09:18:23.471355 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-457fae08-b3fd-4168-a4eb-f9e640da20b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:23.472527 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-457fae08-b3fd-4168-a4eb-f9e640da20b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:23.474028 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1064/2143] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 09:18:23 2022] DELETE /identity/v3/projects/6c350a7b4ae944c188fc8af5a91438f3 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:18:26.298346 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad11979-0fe7-4824-b4d8-067be15da062 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:26.300085 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-7ad11979-0fe7-4824-b4d8-067be15da062 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:18:26.386627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad11979-0fe7-4824-b4d8-067be15da062 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:26.450783 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e38c4f2-4acf-4104-9360-f9b049ed95dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0a8af5cadc4cc1ab73e8eb2a20bb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:26.451415 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e38c4f2-4acf-4104-9360-f9b049ed95dd service nova] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:26.451703 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e38c4f2-4acf-4104-9360-f9b049ed95dd service nova] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:26.451982 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e38c4f2-4acf-4104-9360-f9b049ed95dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:26.453761 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-1e38c4f2-4acf-4104-9360-f9b049ed95dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:18:26.465299 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e38c4f2-4acf-4104-9360-f9b049ed95dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:26.466898 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e38c4f2-4acf-4104-9360-f9b049ed95dd service nova] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:26.477667 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1080/2144] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:18:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:18:31.242052 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88941be7-e899-4b19-8524-62ec683461e8 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:31.242522 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88941be7-e899-4b19-8524-62ec683461e8 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:31.242615 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88941be7-e899-4b19-8524-62ec683461e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:31.292594 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:18:31.301850 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-88941be7-e899-4b19-8524-62ec683461e8 None None] MFA Rules not processed for user `c5b2a0e623c348ad94d4820080677be6`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:18:31.304419 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-88941be7-e899-4b19-8524-62ec683461e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:18:31.367917 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1065/2145] 10.222.0.30 () {58 vars in 1009 bytes} [Sat Oct 15 09:18:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:18:31.416135 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e38c4f2-4acf-4104-9360-f9b049ed95dd service nova] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:31.417569 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-1e38c4f2-4acf-4104-9360-f9b049ed95dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:18:31.422280 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:18:31.509706 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e38c4f2-4acf-4104-9360-f9b049ed95dd service nova] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:31.515528 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dffa3b-7a1f-4637-b514-9233c7a0e1cf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e3009e20e4c49e5a74f7fd793b2c329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:31.516130 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dffa3b-7a1f-4637-b514-9233c7a0e1cf service swift] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:31.516410 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dffa3b-7a1f-4637-b514-9233c7a0e1cf service swift] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:31.516680 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dffa3b-7a1f-4637-b514-9233c7a0e1cf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:31.524904 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dffa3b-7a1f-4637-b514-9233c7a0e1cf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:31.526489 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dffa3b-7a1f-4637-b514-9233c7a0e1cf service swift] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:31.534651 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1081/2146] 10.222.0.30 () {60 vars in 1384 bytes} [Sat Oct 15 09:18:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 09:18:32.035058 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ab99c-81f7-4aac-a65d-e5a330f986b4 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:32.035347 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ab99c-81f7-4aac-a65d-e5a330f986b4 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:32.035625 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ab99c-81f7-4aac-a65d-e5a330f986b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:32.056265 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-c36ab99c-81f7-4aac-a65d-e5a330f986b4 None None] MFA Rules not processed for user `c5b2a0e623c348ad94d4820080677be6`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:18:32.057453 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-c36ab99c-81f7-4aac-a65d-e5a330f986b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:18:32.064454 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1066/2147] 10.222.0.30 () {58 vars in 967 bytes} [Sat Oct 15 09:18:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 09:18:32.076823 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dffa3b-7a1f-4637-b514-9233c7a0e1cf service swift] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:32.087803 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dffa3b-7a1f-4637-b514-9233c7a0e1cf service swift] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:32.092743 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd73655-9c3c-4997-aa68-b8342dc250fb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e3009e20e4c49e5a74f7fd793b2c329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:32.093331 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd73655-9c3c-4997-aa68-b8342dc250fb service swift] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:32.093613 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd73655-9c3c-4997-aa68-b8342dc250fb service swift] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:32.093941 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd73655-9c3c-4997-aa68-b8342dc250fb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:32.102329 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd73655-9c3c-4997-aa68-b8342dc250fb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:32.103888 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd73655-9c3c-4997-aa68-b8342dc250fb service swift] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:32.111899 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1082/2148] 10.222.0.30 () {60 vars in 1384 bytes} [Sat Oct 15 09:18:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 09:18:32.339798 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b91b1-5230-4ee5-b202-be0d2b6baa87 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:32.340248 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b91b1-5230-4ee5-b202-be0d2b6baa87 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:32.340352 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b91b1-5230-4ee5-b202-be0d2b6baa87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:32.359679 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-5b3b91b1-5230-4ee5-b202-be0d2b6baa87 None None] MFA Rules not processed for user `c5b2a0e623c348ad94d4820080677be6`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:18:32.360865 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-5b3b91b1-5230-4ee5-b202-be0d2b6baa87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:18:32.367857 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1067/2149] 10.222.0.30 () {58 vars in 1009 bytes} [Sat Oct 15 09:18:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:18:32.379800 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd73655-9c3c-4997-aa68-b8342dc250fb service swift] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:32.390899 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd73655-9c3c-4997-aa68-b8342dc250fb service swift] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:32.395923 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67284752-438c-4a31-9398-90b74facce62 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e3009e20e4c49e5a74f7fd793b2c329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:32.396516 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67284752-438c-4a31-9398-90b74facce62 service swift] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:32.396844 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67284752-438c-4a31-9398-90b74facce62 service swift] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:32.397606 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67284752-438c-4a31-9398-90b74facce62 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:32.406593 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67284752-438c-4a31-9398-90b74facce62 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:32.408210 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67284752-438c-4a31-9398-90b74facce62 service swift] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:32.416285 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1083/2150] 10.222.0.30 () {60 vars in 1384 bytes} [Sat Oct 15 09:18:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 09:18:32.735971 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196faa82-d859-4201-8c3e-2ab9fb551297 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:32.736254 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196faa82-d859-4201-8c3e-2ab9fb551297 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:32.736557 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196faa82-d859-4201-8c3e-2ab9fb551297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:32.755858 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-196faa82-d859-4201-8c3e-2ab9fb551297 None None] MFA Rules not processed for user `c5b2a0e623c348ad94d4820080677be6`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:18:32.757716 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-196faa82-d859-4201-8c3e-2ab9fb551297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:18:32.764858 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1068/2151] 10.222.0.30 () {58 vars in 967 bytes} [Sat Oct 15 09:18:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:18:32.777011 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67284752-438c-4a31-9398-90b74facce62 service swift] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:32.787459 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67284752-438c-4a31-9398-90b74facce62 service swift] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:32.792441 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dec4d40-e31a-4429-ad9c-a7a659e04157 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e3009e20e4c49e5a74f7fd793b2c329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:32.792971 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dec4d40-e31a-4429-ad9c-a7a659e04157 service swift] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:32.793254 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dec4d40-e31a-4429-ad9c-a7a659e04157 service swift] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:32.793527 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dec4d40-e31a-4429-ad9c-a7a659e04157 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:32.801647 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dec4d40-e31a-4429-ad9c-a7a659e04157 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:32.803764 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dec4d40-e31a-4429-ad9c-a7a659e04157 service swift] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:32.812007 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1084/2152] 10.222.0.30 () {60 vars in 1384 bytes} [Sat Oct 15 09:18:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 09:18:34.871312 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196faa82-d859-4201-8c3e-2ab9fb551297 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:34.872739 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-196faa82-d859-4201-8c3e-2ab9fb551297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:18:34.941225 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196faa82-d859-4201-8c3e-2ab9fb551297 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:34.946414 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99125437-c948-450c-9b84-845768dc9364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:34.947046 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99125437-c948-450c-9b84-845768dc9364 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:34.947578 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99125437-c948-450c-9b84-845768dc9364 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:34.947860 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99125437-c948-450c-9b84-845768dc9364 admin admin] PATH_INFO: `/v3/users/db9d9cc63fec4dac87214047576ea2f8` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:34.948991 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99125437-c948-450c-9b84-845768dc9364 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db9d9cc63fec4dac87214047576ea2f8)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:34.950933 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99125437-c948-450c-9b84-845768dc9364 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:34.996401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-99125437-c948-450c-9b84-845768dc9364 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db9d9cc63fec4dac87214047576ea2f8'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:34.997042 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-99125437-c948-450c-9b84-845768dc9364 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db9d9cc63fec4dac87214047576ea2f8'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:35.008848 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-99125437-c948-450c-9b84-845768dc9364 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db9d9cc63fec4dac87214047576ea2f8'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:35.016195 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-99125437-c948-450c-9b84-845768dc9364 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db9d9cc63fec4dac87214047576ea2f8'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:35.016813 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-99125437-c948-450c-9b84-845768dc9364 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db9d9cc63fec4dac87214047576ea2f8'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:35.024938 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:18:35.030329 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1069/2153] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 09:18:34 2022] DELETE /identity/v3/users/db9d9cc63fec4dac87214047576ea2f8 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:18:35.035081 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092af4fa-765f-44ec-abb3-7faa0ae60f59 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:35.035560 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092af4fa-765f-44ec-abb3-7faa0ae60f59 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:35.035652 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092af4fa-765f-44ec-abb3-7faa0ae60f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:35.056041 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-092af4fa-765f-44ec-abb3-7faa0ae60f59 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:18:35.057242 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-092af4fa-765f-44ec-abb3-7faa0ae60f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:18:35.114536 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1085/2154] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:18:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:18:35.124675 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99125437-c948-450c-9b84-845768dc9364 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:35.126125 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-99125437-c948-450c-9b84-845768dc9364 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:18:35.184078 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99125437-c948-450c-9b84-845768dc9364 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:35.187776 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d555b56-a5d0-443f-b7be-b53aeffa2cce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:35.188388 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d555b56-a5d0-443f-b7be-b53aeffa2cce service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:35.188670 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d555b56-a5d0-443f-b7be-b53aeffa2cce service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:35.188943 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d555b56-a5d0-443f-b7be-b53aeffa2cce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:35.197857 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d555b56-a5d0-443f-b7be-b53aeffa2cce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:35.199294 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d555b56-a5d0-443f-b7be-b53aeffa2cce service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:35.208420 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1070/2155] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:18:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:18:35.370950 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092af4fa-765f-44ec-abb3-7faa0ae60f59 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:35.372363 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-092af4fa-765f-44ec-abb3-7faa0ae60f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:18:35.386382 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092af4fa-765f-44ec-abb3-7faa0ae60f59 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:35.392628 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd55d703-3463-4ba5-860e-32e3c71e506e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:35.393337 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd55d703-3463-4ba5-860e-32e3c71e506e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:35.393545 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd55d703-3463-4ba5-860e-32e3c71e506e admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:35.393868 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd55d703-3463-4ba5-860e-32e3c71e506e admin admin] PATH_INFO: `/v3/projects/9c5258b2dbd7421da1dce3db05030f54` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:35.400528 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd55d703-3463-4ba5-860e-32e3c71e506e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c5258b2dbd7421da1dce3db05030f54)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:35.402592 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd55d703-3463-4ba5-860e-32e3c71e506e admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:35.437513 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:18:35.451840 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-fd55d703-3463-4ba5-860e-32e3c71e506e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c5258b2dbd7421da1dce3db05030f54', 'request_id': 'req-fd55d703-3463-4ba5-860e-32e3c71e506e'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:35.458111 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-fd55d703-3463-4ba5-860e-32e3c71e506e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c5258b2dbd7421da1dce3db05030f54', 'request_id': 'req-fd55d703-3463-4ba5-860e-32e3c71e506e'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:35.465932 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-fd55d703-3463-4ba5-860e-32e3c71e506e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c5258b2dbd7421da1dce3db05030f54', 'request_id': 'req-fd55d703-3463-4ba5-860e-32e3c71e506e'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:35.476973 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-fd55d703-3463-4ba5-860e-32e3c71e506e admin admin] The token cache is being invalidate because project 9c5258b2dbd7421da1dce3db05030f54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 09:18:35.477267 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-fd55d703-3463-4ba5-860e-32e3c71e506e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:35.477941 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-fd55d703-3463-4ba5-860e-32e3c71e506e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:18:35.479453 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1086/2156] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 09:18:35 2022] DELETE /identity/v3/projects/9c5258b2dbd7421da1dce3db05030f54 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:18:35.484465 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ed6af-574f-46e0-aaab-2f2affa20d61 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:35.484750 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ed6af-574f-46e0-aaab-2f2affa20d61 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:35.485024 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ed6af-574f-46e0-aaab-2f2affa20d61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:35.504225 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-699ed6af-574f-46e0-aaab-2f2affa20d61 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:18:35.505419 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-699ed6af-574f-46e0-aaab-2f2affa20d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:18:35.563000 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1071/2157] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:18:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:18:35.571041 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73b2309-dc1e-4056-b5be-5846df5e9029 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:35.571356 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73b2309-dc1e-4056-b5be-5846df5e9029 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:35.571661 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73b2309-dc1e-4056-b5be-5846df5e9029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:35.591405 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-e73b2309-dc1e-4056-b5be-5846df5e9029 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:18:35.592591 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-e73b2309-dc1e-4056-b5be-5846df5e9029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:18:35.600961 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1087/2158] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:18:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:18:35.604984 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699ed6af-574f-46e0-aaab-2f2affa20d61 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:35.614715 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699ed6af-574f-46e0-aaab-2f2affa20d61 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:35.619424 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9f2029-fa4a-4301-8bdc-623e01d2a7a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:35.620028 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f2029-fa4a-4301-8bdc-623e01d2a7a6 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:35.620310 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f2029-fa4a-4301-8bdc-623e01d2a7a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:35.620586 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f2029-fa4a-4301-8bdc-623e01d2a7a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:35.621231 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9f2029-fa4a-4301-8bdc-623e01d2a7a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 09:18:35.621537 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9f2029-fa4a-4301-8bdc-623e01d2a7a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:35.623247 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9f2029-fa4a-4301-8bdc-623e01d2a7a6 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:35.637521 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1072/2159] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 09:18:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 09:18:35.639318 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73b2309-dc1e-4056-b5be-5846df5e9029 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:35.649860 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73b2309-dc1e-4056-b5be-5846df5e9029 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:35.654767 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f1f23-3e96-4a3e-8883-33092d2758b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:35.655363 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f1f23-3e96-4a3e-8883-33092d2758b1 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:35.655644 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f1f23-3e96-4a3e-8883-33092d2758b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:35.655915 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f1f23-3e96-4a3e-8883-33092d2758b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:35.656580 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293f1f23-3e96-4a3e-8883-33092d2758b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:35.658519 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293f1f23-3e96-4a3e-8883-33092d2758b1 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:35.686954 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1088/2160] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 09:18:35 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:18:35.691040 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9f2029-fa4a-4301-8bdc-623e01d2a7a6 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:35.702082 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9f2029-fa4a-4301-8bdc-623e01d2a7a6 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:35.758173 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2147702d-5694-4010-95ca-685776ab092a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:35.758768 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2147702d-5694-4010-95ca-685776ab092a admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:35.759048 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2147702d-5694-4010-95ca-685776ab092a admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:35.759319 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2147702d-5694-4010-95ca-685776ab092a admin admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:35.759951 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2147702d-5694-4010-95ca-685776ab092a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:35.761879 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2147702d-5694-4010-95ca-685776ab092a admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:35.782057 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1073/2161] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 09:18:35 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:18:35.786136 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f1f23-3e96-4a3e-8883-33092d2758b1 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:35.796918 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f1f23-3e96-4a3e-8883-33092d2758b1 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:35.801567 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19139ef-eb42-4029-af56-53391be468e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:35.802206 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19139ef-eb42-4029-af56-53391be468e0 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:35.802489 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19139ef-eb42-4029-af56-53391be468e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:35.802760 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19139ef-eb42-4029-af56-53391be468e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:35.804007 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19139ef-eb42-4029-af56-53391be468e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:35.805623 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19139ef-eb42-4029-af56-53391be468e0 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:35.815301 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1089/2162] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:18:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:18:35.819131 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2147702d-5694-4010-95ca-685776ab092a admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:35.828855 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2147702d-5694-4010-95ca-685776ab092a admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:35.833405 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5e5283-836b-4b10-ae12-de7c1dc1c86c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:35.834024 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5e5283-836b-4b10-ae12-de7c1dc1c86c admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:35.834335 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5e5283-836b-4b10-ae12-de7c1dc1c86c admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:35.834611 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5e5283-836b-4b10-ae12-de7c1dc1c86c admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:35.835248 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5e5283-836b-4b10-ae12-de7c1dc1c86c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:35.836840 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5e5283-836b-4b10-ae12-de7c1dc1c86c admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:35.852817 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1074/2163] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:18:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:18:35.856169 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19139ef-eb42-4029-af56-53391be468e0 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:35.866754 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19139ef-eb42-4029-af56-53391be468e0 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:35.871451 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1620e5-2bc3-4dab-a344-23017eb99d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:35.872037 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1620e5-2bc3-4dab-a344-23017eb99d1d admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:35.872320 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1620e5-2bc3-4dab-a344-23017eb99d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:35.872599 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1620e5-2bc3-4dab-a344-23017eb99d1d admin admin] PATH_INFO: `/v3/projects/6c1e74e55af54016b9e63666c1594675/users/440ce33b27fa46f6aaeab9184d8ef5a3/roles/3248f0268738489b883041851d681bb7` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:35.886948 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1620e5-2bc3-4dab-a344-23017eb99d1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c1e74e55af54016b9e63666c1594675, user_id=440ce33b27fa46f6aaeab9184d8ef5a3, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:35.889561 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1620e5-2bc3-4dab-a344-23017eb99d1d admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:35.902521 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1090/2164] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:18:35 2022] PUT /identity/v3/projects/6c1e74e55af54016b9e63666c1594675/users/440ce33b27fa46f6aaeab9184d8ef5a3/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 09:18:35.907570 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eec2c5-a23a-4099-8257-572d70606f0d None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:35.908078 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eec2c5-a23a-4099-8257-572d70606f0d None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:35.908182 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eec2c5-a23a-4099-8257-572d70606f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:35.928010 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-b7eec2c5-a23a-4099-8257-572d70606f0d None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:18:35.929786 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-b7eec2c5-a23a-4099-8257-572d70606f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:18:35.933809 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:18:35.991776 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1075/2165] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:18:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:18:36.001139 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1620e5-2bc3-4dab-a344-23017eb99d1d admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:36.002845 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-2c1620e5-2bc3-4dab-a344-23017eb99d1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:18:36.058905 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1620e5-2bc3-4dab-a344-23017eb99d1d admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:36.062584 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9d624a-dfd4-450d-bf20-ef7f75a5837c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:36.063181 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9d624a-dfd4-450d-bf20-ef7f75a5837c service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:36.063460 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9d624a-dfd4-450d-bf20-ef7f75a5837c service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:36.063738 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9d624a-dfd4-450d-bf20-ef7f75a5837c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:36.072500 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9d624a-dfd4-450d-bf20-ef7f75a5837c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:36.073941 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9d624a-dfd4-450d-bf20-ef7f75a5837c service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:36.083233 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1091/2166] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:18:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:18:39.986783 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade6bd6-c31f-4287-ac76-00e5aa9d908b None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:39.987337 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade6bd6-c31f-4287-ac76-00e5aa9d908b None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:39.987337 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade6bd6-c31f-4287-ac76-00e5aa9d908b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:40.017641 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-3ade6bd6-c31f-4287-ac76-00e5aa9d908b None None] MFA Rules not processed for user `440ce33b27fa46f6aaeab9184d8ef5a3`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:18:40.019023 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-3ade6bd6-c31f-4287-ac76-00e5aa9d908b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:18:40.094011 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1076/2167] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:18:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:18:40.103833 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9d624a-dfd4-450d-bf20-ef7f75a5837c service neutron] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:40.130403 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9d624a-dfd4-450d-bf20-ef7f75a5837c service neutron] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:40.136080 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d905fa0-d43a-46fe-8e20-760497ae9303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:40.136703 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d905fa0-d43a-46fe-8e20-760497ae9303 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:40.136987 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d905fa0-d43a-46fe-8e20-760497ae9303 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:40.137267 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d905fa0-d43a-46fe-8e20-760497ae9303 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:40.137940 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d905fa0-d43a-46fe-8e20-760497ae9303 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:40.139895 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d905fa0-d43a-46fe-8e20-760497ae9303 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:40.179242 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1092/2168] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 09:18:40 2022] POST /identity/v3/projects => generated 373 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:18:40.183395 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ade6bd6-c31f-4287-ac76-00e5aa9d908b None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:40.195460 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ade6bd6-c31f-4287-ac76-00e5aa9d908b None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:40.293052 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d40420-4ea2-4796-9021-78e06b8bcfd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:40.293706 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d40420-4ea2-4796-9021-78e06b8bcfd3 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:40.293989 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d40420-4ea2-4796-9021-78e06b8bcfd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:40.294257 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d40420-4ea2-4796-9021-78e06b8bcfd3 admin admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:40.294893 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d40420-4ea2-4796-9021-78e06b8bcfd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:40.297106 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d40420-4ea2-4796-9021-78e06b8bcfd3 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:40.324795 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1077/2169] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 09:18:40 2022] POST /identity/v3/users => generated 293 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:18:40.329222 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d905fa0-d43a-46fe-8e20-760497ae9303 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:40.340131 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d905fa0-d43a-46fe-8e20-760497ae9303 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:40.344751 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075ed2bf-883e-498e-b4e1-edc5de7285e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:40.345560 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ed2bf-883e-498e-b4e1-edc5de7285e0 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:40.346027 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ed2bf-883e-498e-b4e1-edc5de7285e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:40.346303 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ed2bf-883e-498e-b4e1-edc5de7285e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:40.346956 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075ed2bf-883e-498e-b4e1-edc5de7285e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:40.348560 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075ed2bf-883e-498e-b4e1-edc5de7285e0 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:40.360809 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1093/2170] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:18:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:18:40.364927 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d40420-4ea2-4796-9021-78e06b8bcfd3 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:40.375724 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d40420-4ea2-4796-9021-78e06b8bcfd3 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:40.380535 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27549601-c5f5-4c4c-b577-f03bf5e79841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:40.381236 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27549601-c5f5-4c4c-b577-f03bf5e79841 admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:40.381514 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27549601-c5f5-4c4c-b577-f03bf5e79841 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:40.381825 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27549601-c5f5-4c4c-b577-f03bf5e79841 admin admin] PATH_INFO: `/v3/projects/e05b8802303a4d3d8c2fe66e6af0e604/users/b37af484a4464540823d1eedcb14ada9/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:40.404379 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27549601-c5f5-4c4c-b577-f03bf5e79841 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e05b8802303a4d3d8c2fe66e6af0e604, user_id=b37af484a4464540823d1eedcb14ada9, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:40.407356 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27549601-c5f5-4c4c-b577-f03bf5e79841 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:40.420820 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1078/2171] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:18:40 2022] PUT /identity/v3/projects/e05b8802303a4d3d8c2fe66e6af0e604/users/b37af484a4464540823d1eedcb14ada9/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 09:18:40.425087 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075ed2bf-883e-498e-b4e1-edc5de7285e0 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:40.435892 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075ed2bf-883e-498e-b4e1-edc5de7285e0 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:40.456194 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:18:40.497488 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c158f9c-fa82-412b-8742-91d049290b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:40.498139 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c158f9c-fa82-412b-8742-91d049290b65 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:40.498422 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c158f9c-fa82-412b-8742-91d049290b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:40.498703 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c158f9c-fa82-412b-8742-91d049290b65 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:40.499408 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c158f9c-fa82-412b-8742-91d049290b65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:40.501296 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c158f9c-fa82-412b-8742-91d049290b65 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:40.511586 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1094/2172] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:18:40 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:18:40.515780 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27549601-c5f5-4c4c-b577-f03bf5e79841 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:40.524668 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:18:40.529088 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27549601-c5f5-4c4c-b577-f03bf5e79841 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:40.535219 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de90d90-9bf3-4f46-879d-16c943c264d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:40.535887 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de90d90-9bf3-4f46-879d-16c943c264d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:40.536168 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de90d90-9bf3-4f46-879d-16c943c264d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:40.536447 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de90d90-9bf3-4f46-879d-16c943c264d8 admin admin] PATH_INFO: `/v3/projects/e05b8802303a4d3d8c2fe66e6af0e604/users/b37af484a4464540823d1eedcb14ada9/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:40.539047 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de90d90-9bf3-4f46-879d-16c943c264d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e05b8802303a4d3d8c2fe66e6af0e604, user_id=b37af484a4464540823d1eedcb14ada9, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:40.541769 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de90d90-9bf3-4f46-879d-16c943c264d8 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:40.553859 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1079/2173] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:18:40 2022] PUT /identity/v3/projects/e05b8802303a4d3d8c2fe66e6af0e604/users/b37af484a4464540823d1eedcb14ada9/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:18:44.943409 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c628bb6-bc0e-479e-bc03-acb2b8e04355 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:44.943906 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c628bb6-bc0e-479e-bc03-acb2b8e04355 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:44.944015 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c628bb6-bc0e-479e-bc03-acb2b8e04355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:44.975650 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-9c628bb6-bc0e-479e-bc03-acb2b8e04355 None None] MFA Rules not processed for user `b37af484a4464540823d1eedcb14ada9`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:18:44.977117 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-9c628bb6-bc0e-479e-bc03-acb2b8e04355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:18:45.056952 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1095/2174] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:18:44 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:18:45.071471 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de90d90-9bf3-4f46-879d-16c943c264d8 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:18:45.073321 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-9de90d90-9bf3-4f46-879d-16c943c264d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:18:45.194449 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de90d90-9bf3-4f46-879d-16c943c264d8 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:18:45.200305 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a755ca33-9769-4ed8-bea3-40d4b03c4a71 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0a8af5cadc4cc1ab73e8eb2a20bb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:18:45.200926 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a755ca33-9769-4ed8-bea3-40d4b03c4a71 service nova] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:45.201204 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a755ca33-9769-4ed8-bea3-40d4b03c4a71 service nova] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:45.201477 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a755ca33-9769-4ed8-bea3-40d4b03c4a71 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:45.265168 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a755ca33-9769-4ed8-bea3-40d4b03c4a71 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:18:45.266847 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a755ca33-9769-4ed8-bea3-40d4b03c4a71 service nova] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:18:45.277741 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1080/2175] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:18:45 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:18:46.259672 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98edc425-6023-412b-a205-4e567948f22e None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:18:46.260167 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98edc425-6023-412b-a205-4e567948f22e None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:18:46.260294 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98edc425-6023-412b-a205-4e567948f22e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:18:46.279373 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-98edc425-6023-412b-a205-4e567948f22e None None] MFA Rules not processed for user `440ce33b27fa46f6aaeab9184d8ef5a3`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:18:46.280567 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-98edc425-6023-412b-a205-4e567948f22e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:18:46.287392 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1096/2176] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:18:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:18:55.273108 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=59,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:18:56.274569 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=55 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:19:00.844422 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a755ca33-9769-4ed8-bea3-40d4b03c4a71 service nova] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:19:00.845892 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-a755ca33-9769-4ed8-bea3-40d4b03c4a71 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:19:00.896257 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98edc425-6023-412b-a205-4e567948f22e None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:19:00.897664 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-98edc425-6023-412b-a205-4e567948f22e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:19:00.920619 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a755ca33-9769-4ed8-bea3-40d4b03c4a71 service nova] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:19:00.924449 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c313fe1-130f-43c7-a1b2-fa093e3e193d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:19:00.925064 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c313fe1-130f-43c7-a1b2-fa093e3e193d service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:19:00.925345 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c313fe1-130f-43c7-a1b2-fa093e3e193d service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:19:00.925623 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c313fe1-130f-43c7-a1b2-fa093e3e193d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:19:00.927350 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-1c313fe1-130f-43c7-a1b2-fa093e3e193d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:19:00.937605 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c313fe1-130f-43c7-a1b2-fa093e3e193d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:19:00.939080 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c313fe1-130f-43c7-a1b2-fa093e3e193d service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:19:00.947139 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1081/2177] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:19:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:19:00.968972 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98edc425-6023-412b-a205-4e567948f22e None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:19:00.974036 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2b2e26-55a0-4e4e-ace1-3ec228d87ebd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38986ca2612c4fc59d960cd693834f9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:19:00.974677 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b2e26-55a0-4e4e-ace1-3ec228d87ebd service placement] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:19:00.974959 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b2e26-55a0-4e4e-ace1-3ec228d87ebd service placement] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:19:00.975268 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b2e26-55a0-4e4e-ace1-3ec228d87ebd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:19:00.976966 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-bd2b2e26-55a0-4e4e-ace1-3ec228d87ebd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:19:00.989165 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2b2e26-55a0-4e4e-ace1-3ec228d87ebd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:19:00.990766 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2b2e26-55a0-4e4e-ace1-3ec228d87ebd service placement] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:19:01.000707 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1097/2178] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:19:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 09:19:10.943457 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=31 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:19:10.996239 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=32 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:19:16.914025 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c313fe1-130f-43c7-a1b2-fa093e3e193d service neutron] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:19:16.923180 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c313fe1-130f-43c7-a1b2-fa093e3e193d service neutron] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:19:16.927077 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c7979f-090b-486c-9b74-3b7d538e696d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:19:16.927586 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7979f-090b-486c-9b74-3b7d538e696d service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:19:16.927867 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7979f-090b-486c-9b74-3b7d538e696d service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:19:16.928138 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7979f-090b-486c-9b74-3b7d538e696d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:19:16.937920 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c7979f-090b-486c-9b74-3b7d538e696d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:19:16.939357 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c7979f-090b-486c-9b74-3b7d538e696d service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:19:16.949063 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1082/2179] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:19:16 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:19:26.944556 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=6 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:19:34.612850 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2b2e26-55a0-4e4e-ace1-3ec228d87ebd service placement] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:19:34.614322 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-bd2b2e26-55a0-4e4e-ace1-3ec228d87ebd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:19:34.668854 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2b2e26-55a0-4e4e-ace1-3ec228d87ebd service placement] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:19:34.672546 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5a9d81-bd44-4fa1-9ab8-bcc9f6d592a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:19:34.673130 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a9d81-bd44-4fa1-9ab8-bcc9f6d592a2 service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:19:34.673408 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a9d81-bd44-4fa1-9ab8-bcc9f6d592a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:19:34.673710 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a9d81-bd44-4fa1-9ab8-bcc9f6d592a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:19:34.675445 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-ef5a9d81-bd44-4fa1-9ab8-bcc9f6d592a2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:19:34.688889 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5a9d81-bd44-4fa1-9ab8-bcc9f6d592a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:19:34.690340 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5a9d81-bd44-4fa1-9ab8-bcc9f6d592a2 service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:19:34.701611 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1098/2180] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:19:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:19:44.696148 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=26 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:19:48.542489 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c7979f-090b-486c-9b74-3b7d538e696d service neutron] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:19:48.543931 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-a5c7979f-090b-486c-9b74-3b7d538e696d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:19:48.608211 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c7979f-090b-486c-9b74-3b7d538e696d service neutron] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:19:48.612707 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa19a4f-2d37-410a-ad2b-bf5b44d8b010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:19:48.613327 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa19a4f-2d37-410a-ad2b-bf5b44d8b010 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:19:48.613607 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa19a4f-2d37-410a-ad2b-bf5b44d8b010 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:19:48.613918 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa19a4f-2d37-410a-ad2b-bf5b44d8b010 admin admin] PATH_INFO: `/v3/users/525f7c26e8e74e17bf07d5d62150ad56` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:19:48.615018 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa19a4f-2d37-410a-ad2b-bf5b44d8b010 admin admin] RBAC: Authorizing `identity:delete_user(user_id=525f7c26e8e74e17bf07d5d62150ad56)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:19:48.616934 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa19a4f-2d37-410a-ad2b-bf5b44d8b010 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:19:48.662509 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-efa19a4f-2d37-410a-ad2b-bf5b44d8b010 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '525f7c26e8e74e17bf07d5d62150ad56'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:19:48.663092 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-efa19a4f-2d37-410a-ad2b-bf5b44d8b010 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '525f7c26e8e74e17bf07d5d62150ad56'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:19:48.675166 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-efa19a4f-2d37-410a-ad2b-bf5b44d8b010 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '525f7c26e8e74e17bf07d5d62150ad56'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:19:48.681462 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-efa19a4f-2d37-410a-ad2b-bf5b44d8b010 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '525f7c26e8e74e17bf07d5d62150ad56'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:19:48.682680 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-efa19a4f-2d37-410a-ad2b-bf5b44d8b010 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '525f7c26e8e74e17bf07d5d62150ad56'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:19:48.694642 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1083/2181] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 09:19:48 2022] DELETE /identity/v3/users/525f7c26e8e74e17bf07d5d62150ad56 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 09:19:48.699619 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99404d0d-a4b6-4811-b95b-7c9fd7a287a0 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:19:48.700121 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99404d0d-a4b6-4811-b95b-7c9fd7a287a0 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:19:48.700211 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99404d0d-a4b6-4811-b95b-7c9fd7a287a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:19:48.719239 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-99404d0d-a4b6-4811-b95b-7c9fd7a287a0 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:19:48.720430 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-99404d0d-a4b6-4811-b95b-7c9fd7a287a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:19:48.777475 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1099/2182] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:19:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:19:48.787129 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa19a4f-2d37-410a-ad2b-bf5b44d8b010 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:19:48.788519 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-efa19a4f-2d37-410a-ad2b-bf5b44d8b010 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:19:48.844555 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa19a4f-2d37-410a-ad2b-bf5b44d8b010 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:19:48.848121 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0366b5a-4ce9-437d-840b-e1af740b63bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:19:48.848709 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0366b5a-4ce9-437d-840b-e1af740b63bd service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:19:48.848987 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0366b5a-4ce9-437d-840b-e1af740b63bd service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:19:48.849264 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0366b5a-4ce9-437d-840b-e1af740b63bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:19:48.859187 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0366b5a-4ce9-437d-840b-e1af740b63bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:19:48.860610 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0366b5a-4ce9-437d-840b-e1af740b63bd service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:19:48.870402 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1084/2183] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:19:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:19:49.135958 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99404d0d-a4b6-4811-b95b-7c9fd7a287a0 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:19:49.137477 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-99404d0d-a4b6-4811-b95b-7c9fd7a287a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:19:49.149264 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99404d0d-a4b6-4811-b95b-7c9fd7a287a0 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:19:49.154627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7db7bf-885e-4c6f-9b27-333c47b517c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:19:49.155251 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7db7bf-885e-4c6f-9b27-333c47b517c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:19:49.155533 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7db7bf-885e-4c6f-9b27-333c47b517c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:19:49.155835 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7db7bf-885e-4c6f-9b27-333c47b517c0 admin admin] PATH_INFO: `/v3/projects/0ade5da44ad948fdb77558898be6bd38` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:19:49.156989 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7db7bf-885e-4c6f-9b27-333c47b517c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ade5da44ad948fdb77558898be6bd38)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:19:49.158923 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7db7bf-885e-4c6f-9b27-333c47b517c0 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:19:49.205066 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-ef7db7bf-885e-4c6f-9b27-333c47b517c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ade5da44ad948fdb77558898be6bd38', 'request_id': 'req-ef7db7bf-885e-4c6f-9b27-333c47b517c0'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:19:49.211247 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-ef7db7bf-885e-4c6f-9b27-333c47b517c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ade5da44ad948fdb77558898be6bd38', 'request_id': 'req-ef7db7bf-885e-4c6f-9b27-333c47b517c0'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:19:49.218576 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-ef7db7bf-885e-4c6f-9b27-333c47b517c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ade5da44ad948fdb77558898be6bd38', 'request_id': 'req-ef7db7bf-885e-4c6f-9b27-333c47b517c0'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:19:49.229281 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-ef7db7bf-885e-4c6f-9b27-333c47b517c0 admin admin] The token cache is being invalidate because project 0ade5da44ad948fdb77558898be6bd38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 09:19:49.229575 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-ef7db7bf-885e-4c6f-9b27-333c47b517c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:19:49.230230 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-ef7db7bf-885e-4c6f-9b27-333c47b517c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:19:49.231715 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1100/2184] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 09:19:49 2022] DELETE /identity/v3/projects/0ade5da44ad948fdb77558898be6bd38 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:19:49.237452 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e2f345-0c9d-4602-9e9b-30fec6815160 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:19:49.237769 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e2f345-0c9d-4602-9e9b-30fec6815160 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:19:49.238046 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e2f345-0c9d-4602-9e9b-30fec6815160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:19:49.257813 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-49e2f345-0c9d-4602-9e9b-30fec6815160 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:19:49.259010 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-49e2f345-0c9d-4602-9e9b-30fec6815160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:19:49.275235 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:19:49.323905 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1085/2185] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:19:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 09:19:49.331485 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33535998-cfa7-4604-8196-a6796c4867d5 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:19:49.331775 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33535998-cfa7-4604-8196-a6796c4867d5 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:19:49.332058 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33535998-cfa7-4604-8196-a6796c4867d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:19:49.351364 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-33535998-cfa7-4604-8196-a6796c4867d5 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:19:49.352564 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-33535998-cfa7-4604-8196-a6796c4867d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:19:49.360978 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1101/2186] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:19:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:19:49.365067 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e2f345-0c9d-4602-9e9b-30fec6815160 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:19:49.375286 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e2f345-0c9d-4602-9e9b-30fec6815160 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:19:49.379880 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948ee083-220f-4f0b-b373-67a0b589f789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:19:49.380471 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948ee083-220f-4f0b-b373-67a0b589f789 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:19:49.380752 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948ee083-220f-4f0b-b373-67a0b589f789 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:19:49.381024 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948ee083-220f-4f0b-b373-67a0b589f789 admin admin] PATH_INFO: `/v3/domains` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:19:49.381659 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948ee083-220f-4f0b-b373-67a0b589f789 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 09:19:49.381991 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948ee083-220f-4f0b-b373-67a0b589f789 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:19:49.383595 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948ee083-220f-4f0b-b373-67a0b589f789 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:19:49.395105 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1086/2187] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 09:19:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 09:19:49.399360 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33535998-cfa7-4604-8196-a6796c4867d5 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:19:49.408516 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33535998-cfa7-4604-8196-a6796c4867d5 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:19:49.413275 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65949f80-0554-4a2a-adec-406c00a88290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:19:49.413902 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65949f80-0554-4a2a-adec-406c00a88290 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:19:49.414186 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65949f80-0554-4a2a-adec-406c00a88290 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:19:49.414462 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65949f80-0554-4a2a-adec-406c00a88290 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:19:49.415107 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65949f80-0554-4a2a-adec-406c00a88290 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:19:49.417616 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65949f80-0554-4a2a-adec-406c00a88290 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:19:49.445118 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1102/2188] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 09:19:49 2022] POST /identity/v3/projects => generated 405 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:19:49.449363 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948ee083-220f-4f0b-b373-67a0b589f789 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:19:49.459197 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948ee083-220f-4f0b-b373-67a0b589f789 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:19:49.514768 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc06cb8-720f-4ef2-8bfb-b739f56c75b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:19:49.515370 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc06cb8-720f-4ef2-8bfb-b739f56c75b8 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:19:49.515655 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc06cb8-720f-4ef2-8bfb-b739f56c75b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:19:49.515933 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc06cb8-720f-4ef2-8bfb-b739f56c75b8 admin admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:19:49.516567 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc06cb8-720f-4ef2-8bfb-b739f56c75b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:19:49.518522 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc06cb8-720f-4ef2-8bfb-b739f56c75b8 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:19:49.539504 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1087/2189] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 09:19:49 2022] POST /identity/v3/users => generated 303 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:19:49.544249 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65949f80-0554-4a2a-adec-406c00a88290 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:19:49.554837 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65949f80-0554-4a2a-adec-406c00a88290 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:19:49.559899 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f18530e-6826-4b77-a7c1-13be20723835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:19:49.560492 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f18530e-6826-4b77-a7c1-13be20723835 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:19:49.560800 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f18530e-6826-4b77-a7c1-13be20723835 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:19:49.561125 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f18530e-6826-4b77-a7c1-13be20723835 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:19:49.561800 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f18530e-6826-4b77-a7c1-13be20723835 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:19:49.563484 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f18530e-6826-4b77-a7c1-13be20723835 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:19:49.577241 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1103/2190] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:19:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:19:49.578593 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc06cb8-720f-4ef2-8bfb-b739f56c75b8 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:19:49.589044 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc06cb8-720f-4ef2-8bfb-b739f56c75b8 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:19:49.593667 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b519c60f-78f4-4cca-b816-aaf6385d9999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:19:49.594322 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b519c60f-78f4-4cca-b816-aaf6385d9999 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:19:49.594603 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b519c60f-78f4-4cca-b816-aaf6385d9999 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:19:49.594878 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b519c60f-78f4-4cca-b816-aaf6385d9999 admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:19:49.595520 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b519c60f-78f4-4cca-b816-aaf6385d9999 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:19:49.597115 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b519c60f-78f4-4cca-b816-aaf6385d9999 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:19:49.610602 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1088/2191] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:19:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:19:49.611980 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f18530e-6826-4b77-a7c1-13be20723835 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:19:49.622626 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f18530e-6826-4b77-a7c1-13be20723835 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:19:49.627191 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc578c8f-b4a3-4b74-b62f-90aa1cbdacf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:19:49.627837 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc578c8f-b4a3-4b74-b62f-90aa1cbdacf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:19:49.628123 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc578c8f-b4a3-4b74-b62f-90aa1cbdacf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:19:49.628397 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc578c8f-b4a3-4b74-b62f-90aa1cbdacf7 admin admin] PATH_INFO: `/v3/projects/1fe8c645e64d49e485733fbfc9955d86/users/c8211d098293486681f15a01ae885b1e/roles/3248f0268738489b883041851d681bb7` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:19:49.643101 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc578c8f-b4a3-4b74-b62f-90aa1cbdacf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fe8c645e64d49e485733fbfc9955d86, user_id=c8211d098293486681f15a01ae885b1e, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:19:49.645757 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc578c8f-b4a3-4b74-b62f-90aa1cbdacf7 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:19:49.659373 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1104/2192] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:19:49 2022] PUT /identity/v3/projects/1fe8c645e64d49e485733fbfc9955d86/users/c8211d098293486681f15a01ae885b1e/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 09:19:49.664486 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c1c63-b01c-44fa-b9d5-b81c59805b94 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:19:49.664871 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c1c63-b01c-44fa-b9d5-b81c59805b94 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:19:49.665171 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c1c63-b01c-44fa-b9d5-b81c59805b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:19:49.684403 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-0a1c1c63-b01c-44fa-b9d5-b81c59805b94 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:19:49.685581 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-0a1c1c63-b01c-44fa-b9d5-b81c59805b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:19:49.742202 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1089/2193] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:19:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:19:49.752118 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc578c8f-b4a3-4b74-b62f-90aa1cbdacf7 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:19:49.753927 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-cc578c8f-b4a3-4b74-b62f-90aa1cbdacf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:19:49.776391 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:19:49.813872 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc578c8f-b4a3-4b74-b62f-90aa1cbdacf7 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:19:49.817601 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ffed76-0060-4d4e-bcbd-c082519a4f0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:19:49.818248 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ffed76-0060-4d4e-bcbd-c082519a4f0d service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:19:49.818527 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ffed76-0060-4d4e-bcbd-c082519a4f0d service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:19:49.818798 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ffed76-0060-4d4e-bcbd-c082519a4f0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:19:49.828192 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ffed76-0060-4d4e-bcbd-c082519a4f0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:19:49.829633 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ffed76-0060-4d4e-bcbd-c082519a4f0d service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:19:49.839426 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1105/2194] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:19:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:19:50.607599 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c9c461-6023-4765-8198-b649f58dd0a7 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:19:50.608117 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c9c461-6023-4765-8198-b649f58dd0a7 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:19:50.608232 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c9c461-6023-4765-8198-b649f58dd0a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:19:50.645586 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-44c9c461-6023-4765-8198-b649f58dd0a7 None None] MFA Rules not processed for user `c8211d098293486681f15a01ae885b1e`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:19:50.646859 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-44c9c461-6023-4765-8198-b649f58dd0a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:19:50.667731 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:19:50.718742 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1090/2195] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:19:50 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:19:50.723166 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ffed76-0060-4d4e-bcbd-c082519a4f0d service neutron] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:19:50.733207 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ffed76-0060-4d4e-bcbd-c082519a4f0d service neutron] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:19:50.738000 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b003f3-b1ea-4d4e-82b9-c1af5a76755d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:19:50.738597 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b003f3-b1ea-4d4e-82b9-c1af5a76755d admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:19:50.738877 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b003f3-b1ea-4d4e-82b9-c1af5a76755d admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:19:50.739151 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b003f3-b1ea-4d4e-82b9-c1af5a76755d admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:19:50.739788 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b003f3-b1ea-4d4e-82b9-c1af5a76755d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:19:50.741814 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b003f3-b1ea-4d4e-82b9-c1af5a76755d admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:19:50.773591 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1106/2196] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 09:19:50 2022] POST /identity/v3/projects => generated 399 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 09:19:50.776276 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c9c461-6023-4765-8198-b649f58dd0a7 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:19:50.788467 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c9c461-6023-4765-8198-b649f58dd0a7 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:19:50.844074 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688546bd-9e3e-4c45-9969-fc1eeebe5b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:19:50.844611 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688546bd-9e3e-4c45-9969-fc1eeebe5b12 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:19:50.844896 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688546bd-9e3e-4c45-9969-fc1eeebe5b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:19:50.845174 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688546bd-9e3e-4c45-9969-fc1eeebe5b12 admin admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:19:50.845850 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688546bd-9e3e-4c45-9969-fc1eeebe5b12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:19:50.847785 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688546bd-9e3e-4c45-9969-fc1eeebe5b12 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:19:50.868799 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1091/2197] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 09:19:50 2022] POST /identity/v3/users => generated 306 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:19:50.872836 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b003f3-b1ea-4d4e-82b9-c1af5a76755d admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:19:50.885462 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b003f3-b1ea-4d4e-82b9-c1af5a76755d admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:19:50.890245 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c84752-183b-4957-96cb-0b0862ce83ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:19:50.890858 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c84752-183b-4957-96cb-0b0862ce83ca admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:19:50.891138 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c84752-183b-4957-96cb-0b0862ce83ca admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:19:50.891416 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c84752-183b-4957-96cb-0b0862ce83ca admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:19:50.892062 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c84752-183b-4957-96cb-0b0862ce83ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:19:50.893651 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c84752-183b-4957-96cb-0b0862ce83ca admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:19:50.905058 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1107/2198] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:19:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:19:50.909120 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688546bd-9e3e-4c45-9969-fc1eeebe5b12 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:19:50.919129 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688546bd-9e3e-4c45-9969-fc1eeebe5b12 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:19:50.924043 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8400a7-ea84-494c-a809-8fca56b4137d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:19:50.924689 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8400a7-ea84-494c-a809-8fca56b4137d admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:19:50.924973 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8400a7-ea84-494c-a809-8fca56b4137d admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:19:50.925249 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8400a7-ea84-494c-a809-8fca56b4137d admin admin] PATH_INFO: `/v3/projects/e5c50557d6ec4aa9881bb005ead8dbc8/users/ffc4f103df0e4427b158d2cbf91d2d24/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:19:50.940200 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8400a7-ea84-494c-a809-8fca56b4137d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5c50557d6ec4aa9881bb005ead8dbc8, user_id=ffc4f103df0e4427b158d2cbf91d2d24, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:19:50.942888 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8400a7-ea84-494c-a809-8fca56b4137d admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:19:50.956035 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1092/2199] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:19:50 2022] PUT /identity/v3/projects/e5c50557d6ec4aa9881bb005ead8dbc8/users/ffc4f103df0e4427b158d2cbf91d2d24/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:19:50.959951 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c84752-183b-4957-96cb-0b0862ce83ca admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:19:50.969922 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c84752-183b-4957-96cb-0b0862ce83ca admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:19:51.028402 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb3d762-e058-4f79-a286-7d9ce30dd6b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:19:51.029045 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb3d762-e058-4f79-a286-7d9ce30dd6b1 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:19:51.029324 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb3d762-e058-4f79-a286-7d9ce30dd6b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:19:51.029601 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb3d762-e058-4f79-a286-7d9ce30dd6b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:19:51.030301 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb3d762-e058-4f79-a286-7d9ce30dd6b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:19:51.031886 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb3d762-e058-4f79-a286-7d9ce30dd6b1 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:19:51.041748 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1108/2200] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:19:50 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:19:51.046300 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8400a7-ea84-494c-a809-8fca56b4137d admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:19:51.057906 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8400a7-ea84-494c-a809-8fca56b4137d admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:19:51.062667 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85970981-c983-47ae-a35c-82f05f4a00d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:19:51.063329 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85970981-c983-47ae-a35c-82f05f4a00d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:19:51.063610 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85970981-c983-47ae-a35c-82f05f4a00d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:19:51.063888 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85970981-c983-47ae-a35c-82f05f4a00d9 admin admin] PATH_INFO: `/v3/projects/e5c50557d6ec4aa9881bb005ead8dbc8/users/ffc4f103df0e4427b158d2cbf91d2d24/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:19:51.065890 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85970981-c983-47ae-a35c-82f05f4a00d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5c50557d6ec4aa9881bb005ead8dbc8, user_id=ffc4f103df0e4427b158d2cbf91d2d24, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:19:51.068551 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85970981-c983-47ae-a35c-82f05f4a00d9 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:19:51.079750 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1093/2201] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:19:51 2022] PUT /identity/v3/projects/e5c50557d6ec4aa9881bb005ead8dbc8/users/ffc4f103df0e4427b158d2cbf91d2d24/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:19:51.705827 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd6cfb4-c3bf-4202-900c-00fad7d1a548 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:19:51.706358 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd6cfb4-c3bf-4202-900c-00fad7d1a548 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:19:51.706714 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd6cfb4-c3bf-4202-900c-00fad7d1a548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:19:51.740847 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-3fd6cfb4-c3bf-4202-900c-00fad7d1a548 None None] MFA Rules not processed for user `ffc4f103df0e4427b158d2cbf91d2d24`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:19:51.742267 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-3fd6cfb4-c3bf-4202-900c-00fad7d1a548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:19:51.814520 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1109/2202] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:19:51 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:19:52.256471 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85970981-c983-47ae-a35c-82f05f4a00d9 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:19:52.258220 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-85970981-c983-47ae-a35c-82f05f4a00d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:19:52.335404 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85970981-c983-47ae-a35c-82f05f4a00d9 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:19:52.340463 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fed8ff0-d8a1-461f-be43-664a3315f5a8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38986ca2612c4fc59d960cd693834f9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:19:52.341100 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed8ff0-d8a1-461f-be43-664a3315f5a8 service placement] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:19:52.341383 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed8ff0-d8a1-461f-be43-664a3315f5a8 service placement] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:19:52.341659 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed8ff0-d8a1-461f-be43-664a3315f5a8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:19:52.343419 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-8fed8ff0-d8a1-461f-be43-664a3315f5a8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:19:52.355779 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fed8ff0-d8a1-461f-be43-664a3315f5a8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:19:52.357369 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fed8ff0-d8a1-461f-be43-664a3315f5a8 service placement] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:19:52.362672 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:19:52.368589 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1094/2203] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:19:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:19:52.760879 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd6cfb4-c3bf-4202-900c-00fad7d1a548 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:19:52.773909 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd6cfb4-c3bf-4202-900c-00fad7d1a548 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:19:52.788758 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d8fdc-f9df-49b7-aded-6bf27db0d820 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38986ca2612c4fc59d960cd693834f9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:19:52.789439 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d8fdc-f9df-49b7-aded-6bf27db0d820 service placement] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:19:52.789749 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d8fdc-f9df-49b7-aded-6bf27db0d820 service placement] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:19:52.790044 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d8fdc-f9df-49b7-aded-6bf27db0d820 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:19:52.791806 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-0f4d8fdc-f9df-49b7-aded-6bf27db0d820 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:19:52.804041 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4d8fdc-f9df-49b7-aded-6bf27db0d820 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:19:52.805628 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4d8fdc-f9df-49b7-aded-6bf27db0d820 service placement] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:19:52.810844 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:19:52.821150 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fed8ff0-d8a1-461f-be43-664a3315f5a8 service placement] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:19:52.830970 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1110/2204] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:19:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:19:52.885851 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fed8ff0-d8a1-461f-be43-664a3315f5a8 service placement] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:19:52.890496 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44522671-dc42-4825-b098-97e1c48329cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:19:52.891122 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44522671-dc42-4825-b098-97e1c48329cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:19:52.891404 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44522671-dc42-4825-b098-97e1c48329cf admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:19:52.891679 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44522671-dc42-4825-b098-97e1c48329cf admin admin] PATH_INFO: `/v3/users/c8211d098293486681f15a01ae885b1e` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:19:52.892786 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44522671-dc42-4825-b098-97e1c48329cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8211d098293486681f15a01ae885b1e)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:19:52.894754 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44522671-dc42-4825-b098-97e1c48329cf admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:19:52.942050 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-44522671-dc42-4825-b098-97e1c48329cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8211d098293486681f15a01ae885b1e'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:19:52.942709 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-44522671-dc42-4825-b098-97e1c48329cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8211d098293486681f15a01ae885b1e'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:19:52.954995 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-44522671-dc42-4825-b098-97e1c48329cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8211d098293486681f15a01ae885b1e'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:19:52.962027 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-44522671-dc42-4825-b098-97e1c48329cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8211d098293486681f15a01ae885b1e'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:19:52.962593 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-44522671-dc42-4825-b098-97e1c48329cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8211d098293486681f15a01ae885b1e'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:19:52.977077 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1095/2205] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 09:19:52 2022] DELETE /identity/v3/users/c8211d098293486681f15a01ae885b1e => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 09:19:53.155175 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d8fdc-f9df-49b7-aded-6bf27db0d820 service placement] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:19:53.166579 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d8fdc-f9df-49b7-aded-6bf27db0d820 service placement] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:19:53.223618 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72af0b83-cfbe-4067-839d-c1f167b02ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:19:53.224177 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72af0b83-cfbe-4067-839d-c1f167b02ddd admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:19:53.224458 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72af0b83-cfbe-4067-839d-c1f167b02ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:19:53.224736 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72af0b83-cfbe-4067-839d-c1f167b02ddd admin admin] PATH_INFO: `/v3/projects/1fe8c645e64d49e485733fbfc9955d86` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:19:53.225892 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72af0b83-cfbe-4067-839d-c1f167b02ddd admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fe8c645e64d49e485733fbfc9955d86)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:19:53.227819 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72af0b83-cfbe-4067-839d-c1f167b02ddd admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:19:53.273092 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-72af0b83-cfbe-4067-839d-c1f167b02ddd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fe8c645e64d49e485733fbfc9955d86', 'request_id': 'req-72af0b83-cfbe-4067-839d-c1f167b02ddd'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:19:53.280315 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-72af0b83-cfbe-4067-839d-c1f167b02ddd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fe8c645e64d49e485733fbfc9955d86', 'request_id': 'req-72af0b83-cfbe-4067-839d-c1f167b02ddd'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:19:53.287471 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-72af0b83-cfbe-4067-839d-c1f167b02ddd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fe8c645e64d49e485733fbfc9955d86', 'request_id': 'req-72af0b83-cfbe-4067-839d-c1f167b02ddd'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:19:53.298586 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-72af0b83-cfbe-4067-839d-c1f167b02ddd admin admin] The token cache is being invalidate because project 1fe8c645e64d49e485733fbfc9955d86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 09:19:53.298871 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-72af0b83-cfbe-4067-839d-c1f167b02ddd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:19:53.299482 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-72af0b83-cfbe-4067-839d-c1f167b02ddd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:19:53.300941 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1111/2206] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 09:19:53 2022] DELETE /identity/v3/projects/1fe8c645e64d49e485733fbfc9955d86 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 09:19:53.305346 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44522671-dc42-4825-b098-97e1c48329cf admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:19:53.306902 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-44522671-dc42-4825-b098-97e1c48329cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:19:53.319738 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44522671-dc42-4825-b098-97e1c48329cf admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:19:53.374726 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c95ff0-18d6-4c87-beb7-d70483645eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:19:53.375350 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c95ff0-18d6-4c87-beb7-d70483645eb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:19:53.375632 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c95ff0-18d6-4c87-beb7-d70483645eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:19:53.375910 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c95ff0-18d6-4c87-beb7-d70483645eb9 admin admin] PATH_INFO: `/v3/users/ffc4f103df0e4427b158d2cbf91d2d24` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:19:53.377041 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c95ff0-18d6-4c87-beb7-d70483645eb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffc4f103df0e4427b158d2cbf91d2d24)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:19:53.379015 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c95ff0-18d6-4c87-beb7-d70483645eb9 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:19:53.401503 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:19:53.425864 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-29c95ff0-18d6-4c87-beb7-d70483645eb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffc4f103df0e4427b158d2cbf91d2d24'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:19:53.426516 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-29c95ff0-18d6-4c87-beb7-d70483645eb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffc4f103df0e4427b158d2cbf91d2d24'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:19:53.438566 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-29c95ff0-18d6-4c87-beb7-d70483645eb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffc4f103df0e4427b158d2cbf91d2d24'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:19:53.445647 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-29c95ff0-18d6-4c87-beb7-d70483645eb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffc4f103df0e4427b158d2cbf91d2d24'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:19:53.446290 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-29c95ff0-18d6-4c87-beb7-d70483645eb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffc4f103df0e4427b158d2cbf91d2d24'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:19:53.458502 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1096/2207] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 09:19:53 2022] DELETE /identity/v3/users/ffc4f103df0e4427b158d2cbf91d2d24 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:19:53.626769 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72af0b83-cfbe-4067-839d-c1f167b02ddd admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:19:53.628499 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-72af0b83-cfbe-4067-839d-c1f167b02ddd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:19:53.643970 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72af0b83-cfbe-4067-839d-c1f167b02ddd admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:19:53.701410 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b70bab-4996-476b-b0e5-172a2c9c2622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:19:53.702113 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b70bab-4996-476b-b0e5-172a2c9c2622 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:19:53.702399 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b70bab-4996-476b-b0e5-172a2c9c2622 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:19:53.702673 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b70bab-4996-476b-b0e5-172a2c9c2622 admin admin] PATH_INFO: `/v3/projects/e5c50557d6ec4aa9881bb005ead8dbc8` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:19:53.703787 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b70bab-4996-476b-b0e5-172a2c9c2622 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5c50557d6ec4aa9881bb005ead8dbc8)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:19:53.705699 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b70bab-4996-476b-b0e5-172a2c9c2622 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:19:53.738115 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:19:53.753029 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d5b70bab-4996-476b-b0e5-172a2c9c2622 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5c50557d6ec4aa9881bb005ead8dbc8', 'request_id': 'req-d5b70bab-4996-476b-b0e5-172a2c9c2622'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:19:53.759181 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d5b70bab-4996-476b-b0e5-172a2c9c2622 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5c50557d6ec4aa9881bb005ead8dbc8', 'request_id': 'req-d5b70bab-4996-476b-b0e5-172a2c9c2622'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:19:53.766538 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d5b70bab-4996-476b-b0e5-172a2c9c2622 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5c50557d6ec4aa9881bb005ead8dbc8', 'request_id': 'req-d5b70bab-4996-476b-b0e5-172a2c9c2622'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:19:53.776682 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d5b70bab-4996-476b-b0e5-172a2c9c2622 admin admin] The token cache is being invalidate because project e5c50557d6ec4aa9881bb005ead8dbc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 09:19:53.776969 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d5b70bab-4996-476b-b0e5-172a2c9c2622 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:19:53.777589 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d5b70bab-4996-476b-b0e5-172a2c9c2622 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:19:53.779082 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1112/2208] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 09:19:53 2022] DELETE /identity/v3/projects/e5c50557d6ec4aa9881bb005ead8dbc8 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:19:53.783963 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7851d214-6dc4-4b33-9027-a56219427066 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:19:53.784323 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7851d214-6dc4-4b33-9027-a56219427066 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:19:53.784609 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7851d214-6dc4-4b33-9027-a56219427066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:19:53.805147 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-7851d214-6dc4-4b33-9027-a56219427066 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:19:53.806387 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-7851d214-6dc4-4b33-9027-a56219427066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:19:53.866628 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1097/2209] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:19:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:19:53.874376 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d3779-a4df-4602-978b-d92dda45a075 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:19:53.874853 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d3779-a4df-4602-978b-d92dda45a075 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:19:53.875070 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d3779-a4df-4602-978b-d92dda45a075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:19:53.895101 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-3f0d3779-a4df-4602-978b-d92dda45a075 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:19:53.896319 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-3f0d3779-a4df-4602-978b-d92dda45a075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:19:53.904927 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1113/2210] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:19:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:19:53.909200 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7851d214-6dc4-4b33-9027-a56219427066 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:19:53.919335 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7851d214-6dc4-4b33-9027-a56219427066 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:19:53.924169 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fab9ac6-2a5e-496b-90ed-449d616efc7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:19:53.924759 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fab9ac6-2a5e-496b-90ed-449d616efc7d admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:19:53.925043 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fab9ac6-2a5e-496b-90ed-449d616efc7d admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:19:53.925318 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fab9ac6-2a5e-496b-90ed-449d616efc7d admin admin] PATH_INFO: `/v3/domains` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:19:53.926004 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fab9ac6-2a5e-496b-90ed-449d616efc7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 09:19:53.926309 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fab9ac6-2a5e-496b-90ed-449d616efc7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:19:53.927914 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fab9ac6-2a5e-496b-90ed-449d616efc7d admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:19:53.941186 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1098/2211] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 09:19:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 09:19:53.944848 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0d3779-a4df-4602-978b-d92dda45a075 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:19:53.954108 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0d3779-a4df-4602-978b-d92dda45a075 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:19:53.958808 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fa354a-6a64-4c9f-a7f9-4e45a43cbe04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:19:53.959403 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa354a-6a64-4c9f-a7f9-4e45a43cbe04 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:19:53.959684 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa354a-6a64-4c9f-a7f9-4e45a43cbe04 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:19:53.959958 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa354a-6a64-4c9f-a7f9-4e45a43cbe04 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:19:53.960586 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fa354a-6a64-4c9f-a7f9-4e45a43cbe04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:19:53.962505 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fa354a-6a64-4c9f-a7f9-4e45a43cbe04 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:19:53.989961 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1114/2212] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 09:19:53 2022] POST /identity/v3/projects => generated 381 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:19:53.995852 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fab9ac6-2a5e-496b-90ed-449d616efc7d admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:19:54.005740 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fab9ac6-2a5e-496b-90ed-449d616efc7d admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:19:54.059906 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83266e2-18a2-4cb7-9d17-9f5f657795a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:19:54.060496 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83266e2-18a2-4cb7-9d17-9f5f657795a9 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:19:54.060775 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83266e2-18a2-4cb7-9d17-9f5f657795a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:19:54.061044 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83266e2-18a2-4cb7-9d17-9f5f657795a9 admin admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:19:54.061665 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83266e2-18a2-4cb7-9d17-9f5f657795a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:19:54.063593 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83266e2-18a2-4cb7-9d17-9f5f657795a9 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:19:54.084265 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1099/2213] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 09:19:53 2022] POST /identity/v3/users => generated 291 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 09:19:54.088972 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fa354a-6a64-4c9f-a7f9-4e45a43cbe04 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:19:54.099334 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fa354a-6a64-4c9f-a7f9-4e45a43cbe04 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:19:54.104712 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fd18dc-af58-4c43-9e80-3eefd26b37c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:19:54.105318 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fd18dc-af58-4c43-9e80-3eefd26b37c8 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:19:54.105602 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fd18dc-af58-4c43-9e80-3eefd26b37c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:19:54.105949 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fd18dc-af58-4c43-9e80-3eefd26b37c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:19:54.106590 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fd18dc-af58-4c43-9e80-3eefd26b37c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:19:54.108195 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fd18dc-af58-4c43-9e80-3eefd26b37c8 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:19:54.118200 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1115/2214] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:19:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:19:54.122162 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83266e2-18a2-4cb7-9d17-9f5f657795a9 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:19:54.131911 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83266e2-18a2-4cb7-9d17-9f5f657795a9 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:19:54.136502 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50636c64-1139-442a-93de-841214674cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:19:54.137091 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50636c64-1139-442a-93de-841214674cd2 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:19:54.137376 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50636c64-1139-442a-93de-841214674cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:19:54.137660 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50636c64-1139-442a-93de-841214674cd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:19:54.138325 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50636c64-1139-442a-93de-841214674cd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:19:54.139936 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50636c64-1139-442a-93de-841214674cd2 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:19:54.150838 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1100/2215] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:19:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:19:54.154510 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fd18dc-af58-4c43-9e80-3eefd26b37c8 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:19:54.165203 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fd18dc-af58-4c43-9e80-3eefd26b37c8 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:19:54.170688 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3d189c-5d3b-4070-91b1-b63df73601e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:19:54.171444 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d189c-5d3b-4070-91b1-b63df73601e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:19:54.171792 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d189c-5d3b-4070-91b1-b63df73601e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:19:54.172141 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d189c-5d3b-4070-91b1-b63df73601e3 admin admin] PATH_INFO: `/v3/projects/deaaca1b336d4b9b9ed5d0477d0b7f06/users/3cff4dad0b344252a34b30de060cab51/roles/3248f0268738489b883041851d681bb7` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:19:54.187527 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3d189c-5d3b-4070-91b1-b63df73601e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=deaaca1b336d4b9b9ed5d0477d0b7f06, user_id=3cff4dad0b344252a34b30de060cab51, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:19:54.190227 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3d189c-5d3b-4070-91b1-b63df73601e3 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:19:54.203342 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1116/2216] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:19:54 2022] PUT /identity/v3/projects/deaaca1b336d4b9b9ed5d0477d0b7f06/users/3cff4dad0b344252a34b30de060cab51/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:19:54.213623 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a1bf9-e03e-472f-b3cc-ecc549a172c8 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:19:54.213935 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a1bf9-e03e-472f-b3cc-ecc549a172c8 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:19:54.214236 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a1bf9-e03e-472f-b3cc-ecc549a172c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:19:54.222409 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:19:54.242830 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-bb6a1bf9-e03e-472f-b3cc-ecc549a172c8 None None] MFA Rules not processed for user `3cff4dad0b344252a34b30de060cab51`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:19:54.244107 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-bb6a1bf9-e03e-472f-b3cc-ecc549a172c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:19:54.307430 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1101/2217] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:19:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:19:54.311785 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3d189c-5d3b-4070-91b1-b63df73601e3 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:19:54.373791 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3d189c-5d3b-4070-91b1-b63df73601e3 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:19:54.378376 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a95ac8-d56e-4cc1-923d-40f9fb6b7bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:19:54.378967 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a95ac8-d56e-4cc1-923d-40f9fb6b7bd2 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:19:54.379528 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a95ac8-d56e-4cc1-923d-40f9fb6b7bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:19:54.379811 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a95ac8-d56e-4cc1-923d-40f9fb6b7bd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:19:54.380445 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a95ac8-d56e-4cc1-923d-40f9fb6b7bd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:19:54.382401 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a95ac8-d56e-4cc1-923d-40f9fb6b7bd2 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:19:54.408594 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1117/2218] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 09:19:54 2022] POST /identity/v3/projects => generated 379 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:19:54.412672 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6a1bf9-e03e-472f-b3cc-ecc549a172c8 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:19:54.424110 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6a1bf9-e03e-472f-b3cc-ecc549a172c8 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:19:54.479706 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4858a0a-9510-4f48-91a7-6fcac3cb643f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:19:54.480317 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4858a0a-9510-4f48-91a7-6fcac3cb643f admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:19:54.480596 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4858a0a-9510-4f48-91a7-6fcac3cb643f admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:19:54.480870 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4858a0a-9510-4f48-91a7-6fcac3cb643f admin admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:19:54.481500 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4858a0a-9510-4f48-91a7-6fcac3cb643f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:19:54.483460 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4858a0a-9510-4f48-91a7-6fcac3cb643f admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:19:54.503925 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1102/2219] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 09:19:54 2022] POST /identity/v3/users => generated 296 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:19:54.508020 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a95ac8-d56e-4cc1-923d-40f9fb6b7bd2 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:19:54.518492 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a95ac8-d56e-4cc1-923d-40f9fb6b7bd2 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:19:54.523066 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f78f18a-4351-488d-b74c-86a538f320ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:19:54.523661 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f78f18a-4351-488d-b74c-86a538f320ab admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:19:54.523941 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f78f18a-4351-488d-b74c-86a538f320ab admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:19:54.524236 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f78f18a-4351-488d-b74c-86a538f320ab admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:19:54.524875 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f78f18a-4351-488d-b74c-86a538f320ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:19:54.526485 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f78f18a-4351-488d-b74c-86a538f320ab admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:19:54.536878 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1118/2220] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:19:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:19:54.540532 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4858a0a-9510-4f48-91a7-6fcac3cb643f admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:19:54.550954 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4858a0a-9510-4f48-91a7-6fcac3cb643f admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:19:54.555446 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ceca4b-1b5f-4430-bb75-6e92520b851c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:19:54.556080 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ceca4b-1b5f-4430-bb75-6e92520b851c admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:19:54.556358 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ceca4b-1b5f-4430-bb75-6e92520b851c admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:19:54.557285 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ceca4b-1b5f-4430-bb75-6e92520b851c admin admin] PATH_INFO: `/v3/projects/a5cdf10dd415427abe3a70bae66ea515/users/32c400f9abe0460892d231367903d24b/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:19:54.571541 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ceca4b-1b5f-4430-bb75-6e92520b851c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5cdf10dd415427abe3a70bae66ea515, user_id=32c400f9abe0460892d231367903d24b, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:19:54.574685 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ceca4b-1b5f-4430-bb75-6e92520b851c admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:19:54.587016 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1103/2221] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:19:54 2022] PUT /identity/v3/projects/a5cdf10dd415427abe3a70bae66ea515/users/32c400f9abe0460892d231367903d24b/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:19:54.591460 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f78f18a-4351-488d-b74c-86a538f320ab admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:19:54.601232 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f78f18a-4351-488d-b74c-86a538f320ab admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:19:54.626784 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:19:54.659546 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82434197-e74a-43d9-8ba2-0c618f6fd140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:19:54.660154 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82434197-e74a-43d9-8ba2-0c618f6fd140 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:19:54.660465 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82434197-e74a-43d9-8ba2-0c618f6fd140 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:19:54.660741 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82434197-e74a-43d9-8ba2-0c618f6fd140 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:19:54.661384 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82434197-e74a-43d9-8ba2-0c618f6fd140 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:19:54.663035 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82434197-e74a-43d9-8ba2-0c618f6fd140 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:19:54.672824 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1119/2222] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:19:54 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:19:54.676979 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ceca4b-1b5f-4430-bb75-6e92520b851c admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:19:54.688047 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ceca4b-1b5f-4430-bb75-6e92520b851c admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:19:54.692857 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca84dee2-1aa2-4857-bda0-43e4456ff0e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:19:54.693540 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca84dee2-1aa2-4857-bda0-43e4456ff0e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:19:54.693868 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca84dee2-1aa2-4857-bda0-43e4456ff0e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:19:54.694148 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca84dee2-1aa2-4857-bda0-43e4456ff0e1 admin admin] PATH_INFO: `/v3/projects/a5cdf10dd415427abe3a70bae66ea515/users/32c400f9abe0460892d231367903d24b/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:19:54.696098 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca84dee2-1aa2-4857-bda0-43e4456ff0e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5cdf10dd415427abe3a70bae66ea515, user_id=32c400f9abe0460892d231367903d24b, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:19:54.699380 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca84dee2-1aa2-4857-bda0-43e4456ff0e1 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:19:54.709441 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1104/2223] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:19:54 2022] PUT /identity/v3/projects/a5cdf10dd415427abe3a70bae66ea515/users/32c400f9abe0460892d231367903d24b/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 09:19:54.719070 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79e245-4a53-4c45-ad3b-464c14334b68 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:19:54.719384 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79e245-4a53-4c45-ad3b-464c14334b68 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:19:54.719663 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79e245-4a53-4c45-ad3b-464c14334b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:19:54.747221 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-fa79e245-4a53-4c45-ad3b-464c14334b68 None None] MFA Rules not processed for user `32c400f9abe0460892d231367903d24b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:19:54.748430 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-fa79e245-4a53-4c45-ad3b-464c14334b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:19:54.814647 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1120/2224] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:19:54 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:19:54.826254 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca84dee2-1aa2-4857-bda0-43e4456ff0e1 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:19:54.827908 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-ca84dee2-1aa2-4857-bda0-43e4456ff0e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:19:54.882487 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:19:54.902385 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca84dee2-1aa2-4857-bda0-43e4456ff0e1 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:19:54.906057 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1c6704-c8b9-43d2-943b-2b8d3f200033 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:19:54.906672 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c6704-c8b9-43d2-943b-2b8d3f200033 service cinder] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:19:54.906952 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c6704-c8b9-43d2-943b-2b8d3f200033 service cinder] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:19:54.907227 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c6704-c8b9-43d2-943b-2b8d3f200033 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:19:54.916072 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1c6704-c8b9-43d2-943b-2b8d3f200033 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:19:54.917495 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1c6704-c8b9-43d2-943b-2b8d3f200033 service cinder] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:19:54.927152 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1105/2225] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:19:54 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:19:55.001298 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa79e245-4a53-4c45-ad3b-464c14334b68 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:19:55.011184 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa79e245-4a53-4c45-ad3b-464c14334b68 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:19:55.014709 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3400d2e2-5dd0-4cbb-b533-50d1e4a02a96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:19:55.015298 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3400d2e2-5dd0-4cbb-b533-50d1e4a02a96 service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:19:55.015577 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3400d2e2-5dd0-4cbb-b533-50d1e4a02a96 service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:19:55.015849 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3400d2e2-5dd0-4cbb-b533-50d1e4a02a96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:19:55.071952 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3400d2e2-5dd0-4cbb-b533-50d1e4a02a96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:19:55.073927 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3400d2e2-5dd0-4cbb-b533-50d1e4a02a96 service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:19:55.083040 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1121/2226] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:19:55 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:20:04.905260 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1c6704-c8b9-43d2-943b-2b8d3f200033 service cinder] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:04.965407 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1c6704-c8b9-43d2-943b-2b8d3f200033 service cinder] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:04.969911 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c240d846-b9d4-413d-bc41-991f855b758d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:04.970528 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240d846-b9d4-413d-bc41-991f855b758d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:04.970805 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240d846-b9d4-413d-bc41-991f855b758d admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:04.971098 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240d846-b9d4-413d-bc41-991f855b758d admin admin] PATH_INFO: `/v3/users/3cff4dad0b344252a34b30de060cab51` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:04.972187 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c240d846-b9d4-413d-bc41-991f855b758d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cff4dad0b344252a34b30de060cab51)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:04.974130 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c240d846-b9d4-413d-bc41-991f855b758d admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:05.018972 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-c240d846-b9d4-413d-bc41-991f855b758d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cff4dad0b344252a34b30de060cab51'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:05.019617 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-c240d846-b9d4-413d-bc41-991f855b758d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cff4dad0b344252a34b30de060cab51'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:05.031257 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-c240d846-b9d4-413d-bc41-991f855b758d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cff4dad0b344252a34b30de060cab51'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:05.038438 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-c240d846-b9d4-413d-bc41-991f855b758d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cff4dad0b344252a34b30de060cab51'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:05.039048 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-c240d846-b9d4-413d-bc41-991f855b758d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cff4dad0b344252a34b30de060cab51'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:05.051602 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1106/2227] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 09:20:04 2022] DELETE /identity/v3/users/3cff4dad0b344252a34b30de060cab51 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:20:05.056002 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa55339b-11b9-4a6b-a570-9b8fc6785514 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:05.057114 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa55339b-11b9-4a6b-a570-9b8fc6785514 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:05.057216 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa55339b-11b9-4a6b-a570-9b8fc6785514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:05.076000 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-fa55339b-11b9-4a6b-a570-9b8fc6785514 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:20:05.077175 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-fa55339b-11b9-4a6b-a570-9b8fc6785514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:20:05.120407 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:20:05.135609 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1122/2228] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:20:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:20:05.145308 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c240d846-b9d4-413d-bc41-991f855b758d admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:05.146749 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-c240d846-b9d4-413d-bc41-991f855b758d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:20:05.204935 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c240d846-b9d4-413d-bc41-991f855b758d admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:05.208732 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8a3f04-fad7-4680-8e40-bc4846fbd530 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:05.209342 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8a3f04-fad7-4680-8e40-bc4846fbd530 service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:05.209654 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8a3f04-fad7-4680-8e40-bc4846fbd530 service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:05.209954 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8a3f04-fad7-4680-8e40-bc4846fbd530 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:05.219610 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8a3f04-fad7-4680-8e40-bc4846fbd530 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:05.221031 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8a3f04-fad7-4680-8e40-bc4846fbd530 service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:05.225880 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:20:05.231410 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1107/2229] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:20:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:20:05.502461 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa55339b-11b9-4a6b-a570-9b8fc6785514 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:05.504027 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-fa55339b-11b9-4a6b-a570-9b8fc6785514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:20:05.518499 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa55339b-11b9-4a6b-a570-9b8fc6785514 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:05.524165 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6b9652-4106-4122-aa4f-73199427663f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:05.524787 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6b9652-4106-4122-aa4f-73199427663f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:05.525744 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6b9652-4106-4122-aa4f-73199427663f admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:05.525955 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6b9652-4106-4122-aa4f-73199427663f admin admin] PATH_INFO: `/v3/projects/deaaca1b336d4b9b9ed5d0477d0b7f06` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:05.527051 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6b9652-4106-4122-aa4f-73199427663f admin admin] RBAC: Authorizing `identity:delete_project(project_id=deaaca1b336d4b9b9ed5d0477d0b7f06)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:05.528957 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6b9652-4106-4122-aa4f-73199427663f admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:05.575628 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-2a6b9652-4106-4122-aa4f-73199427663f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'deaaca1b336d4b9b9ed5d0477d0b7f06', 'request_id': 'req-2a6b9652-4106-4122-aa4f-73199427663f'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:05.581892 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-2a6b9652-4106-4122-aa4f-73199427663f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'deaaca1b336d4b9b9ed5d0477d0b7f06', 'request_id': 'req-2a6b9652-4106-4122-aa4f-73199427663f'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:05.590019 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-2a6b9652-4106-4122-aa4f-73199427663f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'deaaca1b336d4b9b9ed5d0477d0b7f06', 'request_id': 'req-2a6b9652-4106-4122-aa4f-73199427663f'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:05.600477 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-2a6b9652-4106-4122-aa4f-73199427663f admin admin] The token cache is being invalidate because project deaaca1b336d4b9b9ed5d0477d0b7f06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 09:20:05.600768 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-2a6b9652-4106-4122-aa4f-73199427663f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:05.601416 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-2a6b9652-4106-4122-aa4f-73199427663f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:05.602921 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1123/2230] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 09:20:05 2022] DELETE /identity/v3/projects/deaaca1b336d4b9b9ed5d0477d0b7f06 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:20:05.606897 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8a3f04-fad7-4680-8e40-bc4846fbd530 service neutron] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:05.616339 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8a3f04-fad7-4680-8e40-bc4846fbd530 service neutron] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:05.674508 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a1e448-703f-4608-9739-debf35df2551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:05.675144 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a1e448-703f-4608-9739-debf35df2551 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:05.675427 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a1e448-703f-4608-9739-debf35df2551 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:05.675707 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a1e448-703f-4608-9739-debf35df2551 admin admin] PATH_INFO: `/v3/users/32c400f9abe0460892d231367903d24b` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:05.676859 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a1e448-703f-4608-9739-debf35df2551 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32c400f9abe0460892d231367903d24b)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:05.678867 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a1e448-703f-4608-9739-debf35df2551 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:05.723753 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-60a1e448-703f-4608-9739-debf35df2551 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32c400f9abe0460892d231367903d24b'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:05.724397 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-60a1e448-703f-4608-9739-debf35df2551 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32c400f9abe0460892d231367903d24b'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:05.736425 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-60a1e448-703f-4608-9739-debf35df2551 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32c400f9abe0460892d231367903d24b'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:05.743447 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-60a1e448-703f-4608-9739-debf35df2551 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32c400f9abe0460892d231367903d24b'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:05.744650 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-60a1e448-703f-4608-9739-debf35df2551 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32c400f9abe0460892d231367903d24b'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:05.756460 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1108/2231] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 09:20:05 2022] DELETE /identity/v3/users/32c400f9abe0460892d231367903d24b => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:20:06.021208 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6b9652-4106-4122-aa4f-73199427663f admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:06.022796 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-2a6b9652-4106-4122-aa4f-73199427663f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:20:06.038767 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6b9652-4106-4122-aa4f-73199427663f admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:06.095775 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c090525-5aa6-411f-8b0b-25af5d4f4014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:06.096379 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c090525-5aa6-411f-8b0b-25af5d4f4014 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:06.096614 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c090525-5aa6-411f-8b0b-25af5d4f4014 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:06.096891 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c090525-5aa6-411f-8b0b-25af5d4f4014 admin admin] PATH_INFO: `/v3/projects/a5cdf10dd415427abe3a70bae66ea515` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:06.098043 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c090525-5aa6-411f-8b0b-25af5d4f4014 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5cdf10dd415427abe3a70bae66ea515)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:06.099964 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c090525-5aa6-411f-8b0b-25af5d4f4014 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:06.144910 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7c090525-5aa6-411f-8b0b-25af5d4f4014 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5cdf10dd415427abe3a70bae66ea515', 'request_id': 'req-7c090525-5aa6-411f-8b0b-25af5d4f4014'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:06.151553 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7c090525-5aa6-411f-8b0b-25af5d4f4014 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5cdf10dd415427abe3a70bae66ea515', 'request_id': 'req-7c090525-5aa6-411f-8b0b-25af5d4f4014'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:06.158826 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7c090525-5aa6-411f-8b0b-25af5d4f4014 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5cdf10dd415427abe3a70bae66ea515', 'request_id': 'req-7c090525-5aa6-411f-8b0b-25af5d4f4014'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:06.169800 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7c090525-5aa6-411f-8b0b-25af5d4f4014 admin admin] The token cache is being invalidate because project a5cdf10dd415427abe3a70bae66ea515 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 09:20:06.170094 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7c090525-5aa6-411f-8b0b-25af5d4f4014 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:06.170729 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7c090525-5aa6-411f-8b0b-25af5d4f4014 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:06.172217 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1124/2232] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 09:20:06 2022] DELETE /identity/v3/projects/a5cdf10dd415427abe3a70bae66ea515 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:20:06.177382 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87979fb-ab98-4b8e-a471-0632cddcfe75 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:06.177892 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87979fb-ab98-4b8e-a471-0632cddcfe75 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:06.178041 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87979fb-ab98-4b8e-a471-0632cddcfe75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:06.199159 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-a87979fb-ab98-4b8e-a471-0632cddcfe75 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:20:06.200371 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-a87979fb-ab98-4b8e-a471-0632cddcfe75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:20:06.258403 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1109/2233] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:20:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:20:06.267715 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05466747-a1e4-49bf-b44f-a74b1517764e None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:06.268008 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05466747-a1e4-49bf-b44f-a74b1517764e None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:06.268314 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05466747-a1e4-49bf-b44f-a74b1517764e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:06.276129 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:20:06.291333 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-05466747-a1e4-49bf-b44f-a74b1517764e None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:20:06.292549 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-05466747-a1e4-49bf-b44f-a74b1517764e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:20:06.300735 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1125/2234] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:20:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:20:06.305074 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87979fb-ab98-4b8e-a471-0632cddcfe75 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:06.315526 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87979fb-ab98-4b8e-a471-0632cddcfe75 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:06.320027 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d33466-588a-4087-8a8c-f3138711edaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:06.320621 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d33466-588a-4087-8a8c-f3138711edaf admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:06.320902 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d33466-588a-4087-8a8c-f3138711edaf admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:06.321176 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d33466-588a-4087-8a8c-f3138711edaf admin admin] PATH_INFO: `/v3/domains` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:06.321861 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d33466-588a-4087-8a8c-f3138711edaf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 09:20:06.322726 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d33466-588a-4087-8a8c-f3138711edaf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:06.324336 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d33466-588a-4087-8a8c-f3138711edaf admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:06.335669 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1110/2235] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 09:20:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 09:20:06.339997 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05466747-a1e4-49bf-b44f-a74b1517764e None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:06.349167 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05466747-a1e4-49bf-b44f-a74b1517764e None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:06.354273 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d20d8ef-8077-4325-ab65-21fa1937d46a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:06.354863 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d20d8ef-8077-4325-ab65-21fa1937d46a admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:06.355140 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d20d8ef-8077-4325-ab65-21fa1937d46a admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:06.355412 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d20d8ef-8077-4325-ab65-21fa1937d46a admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:06.356046 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d20d8ef-8077-4325-ab65-21fa1937d46a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:06.357968 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d20d8ef-8077-4325-ab65-21fa1937d46a admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:06.384281 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1126/2236] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 09:20:06 2022] POST /identity/v3/projects => generated 387 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:20:06.388414 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d33466-588a-4087-8a8c-f3138711edaf admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:06.398725 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d33466-588a-4087-8a8c-f3138711edaf admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:06.402401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:20:06.457557 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8174c5-812f-4dab-a1cf-260d9c426eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:06.458192 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8174c5-812f-4dab-a1cf-260d9c426eab admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:06.458472 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8174c5-812f-4dab-a1cf-260d9c426eab admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:06.458745 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8174c5-812f-4dab-a1cf-260d9c426eab admin admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:06.459379 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8174c5-812f-4dab-a1cf-260d9c426eab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:06.461330 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8174c5-812f-4dab-a1cf-260d9c426eab admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:06.483609 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1111/2237] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 09:20:06 2022] POST /identity/v3/users => generated 294 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:20:06.489960 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d20d8ef-8077-4325-ab65-21fa1937d46a admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:06.500468 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d20d8ef-8077-4325-ab65-21fa1937d46a admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:06.505027 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176865ed-62a8-4639-aad9-c27403aeaac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:06.505624 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176865ed-62a8-4639-aad9-c27403aeaac5 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:06.505934 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176865ed-62a8-4639-aad9-c27403aeaac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:06.506208 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176865ed-62a8-4639-aad9-c27403aeaac5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:06.506848 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176865ed-62a8-4639-aad9-c27403aeaac5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:06.508433 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176865ed-62a8-4639-aad9-c27403aeaac5 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:06.520019 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1127/2238] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:20:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 09:20:06.523868 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8174c5-812f-4dab-a1cf-260d9c426eab admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:06.534206 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8174c5-812f-4dab-a1cf-260d9c426eab admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:06.538700 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86faa647-9cf6-4f82-ae4c-aec275eb36cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:06.539290 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86faa647-9cf6-4f82-ae4c-aec275eb36cb admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:06.539573 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86faa647-9cf6-4f82-ae4c-aec275eb36cb admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:06.539848 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86faa647-9cf6-4f82-ae4c-aec275eb36cb admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:06.540487 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86faa647-9cf6-4f82-ae4c-aec275eb36cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:06.542181 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86faa647-9cf6-4f82-ae4c-aec275eb36cb admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:06.552756 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1112/2239] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:20:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:20:06.556625 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176865ed-62a8-4639-aad9-c27403aeaac5 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:06.566313 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176865ed-62a8-4639-aad9-c27403aeaac5 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:06.570851 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a2c083-f5b6-49d1-9cf8-9d01fd3a9a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:06.571497 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a2c083-f5b6-49d1-9cf8-9d01fd3a9a2d admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:06.571776 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a2c083-f5b6-49d1-9cf8-9d01fd3a9a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:06.572601 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a2c083-f5b6-49d1-9cf8-9d01fd3a9a2d admin admin] PATH_INFO: `/v3/projects/af3a4a0709904160be303f5093d8ba66/users/88efaa7a600c418fa7df9f8efd3a5ba3/roles/3248f0268738489b883041851d681bb7` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:06.586776 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a2c083-f5b6-49d1-9cf8-9d01fd3a9a2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=af3a4a0709904160be303f5093d8ba66, user_id=88efaa7a600c418fa7df9f8efd3a5ba3, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:06.590055 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a2c083-f5b6-49d1-9cf8-9d01fd3a9a2d admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:06.602448 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1128/2240] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:20:06 2022] PUT /identity/v3/projects/af3a4a0709904160be303f5093d8ba66/users/88efaa7a600c418fa7df9f8efd3a5ba3/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:20:06.612494 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66bca5f-80c3-426d-a564-d0f7a4ebd9f9 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:06.612812 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66bca5f-80c3-426d-a564-d0f7a4ebd9f9 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:06.613117 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66bca5f-80c3-426d-a564-d0f7a4ebd9f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:06.640529 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-c66bca5f-80c3-426d-a564-d0f7a4ebd9f9 None None] MFA Rules not processed for user `88efaa7a600c418fa7df9f8efd3a5ba3`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:20:06.641760 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-c66bca5f-80c3-426d-a564-d0f7a4ebd9f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:20:06.706189 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1113/2241] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:20:06 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:20:06.710685 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a2c083-f5b6-49d1-9cf8-9d01fd3a9a2d admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:06.774245 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a2c083-f5b6-49d1-9cf8-9d01fd3a9a2d admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:06.779654 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cc12ab-9663-45d5-940f-e79883ecc530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:06.780258 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc12ab-9663-45d5-940f-e79883ecc530 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:06.780540 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc12ab-9663-45d5-940f-e79883ecc530 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:06.780813 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc12ab-9663-45d5-940f-e79883ecc530 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:06.781452 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cc12ab-9663-45d5-940f-e79883ecc530 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:06.783376 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cc12ab-9663-45d5-940f-e79883ecc530 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:06.809788 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1129/2242] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 09:20:06 2022] POST /identity/v3/projects => generated 389 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:20:06.813817 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66bca5f-80c3-426d-a564-d0f7a4ebd9f9 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:06.824334 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66bca5f-80c3-426d-a564-d0f7a4ebd9f9 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:06.881598 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dca5e1-f60d-44bd-b22d-65967be93d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:06.882234 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dca5e1-f60d-44bd-b22d-65967be93d47 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:06.882510 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dca5e1-f60d-44bd-b22d-65967be93d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:06.882780 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dca5e1-f60d-44bd-b22d-65967be93d47 admin admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:06.883401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dca5e1-f60d-44bd-b22d-65967be93d47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:06.885310 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dca5e1-f60d-44bd-b22d-65967be93d47 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:06.915274 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1114/2243] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 09:20:06 2022] POST /identity/v3/users => generated 301 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:20:06.915396 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cc12ab-9663-45d5-940f-e79883ecc530 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:06.921268 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cc12ab-9663-45d5-940f-e79883ecc530 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:06.926081 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b34dac-3c84-4c30-9c14-3e2fb2f35cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:06.926674 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b34dac-3c84-4c30-9c14-3e2fb2f35cf7 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:06.926956 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b34dac-3c84-4c30-9c14-3e2fb2f35cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:06.927230 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b34dac-3c84-4c30-9c14-3e2fb2f35cf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:06.927869 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b34dac-3c84-4c30-9c14-3e2fb2f35cf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:06.929459 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b34dac-3c84-4c30-9c14-3e2fb2f35cf7 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:06.939782 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1130/2244] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:20:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:20:06.943568 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dca5e1-f60d-44bd-b22d-65967be93d47 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:06.949791 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:20:06.953465 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dca5e1-f60d-44bd-b22d-65967be93d47 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:06.959429 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a902a7c-5f56-4c9e-b0b9-bc36a234d658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:06.960126 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a902a7c-5f56-4c9e-b0b9-bc36a234d658 admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:06.960414 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a902a7c-5f56-4c9e-b0b9-bc36a234d658 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:06.960692 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a902a7c-5f56-4c9e-b0b9-bc36a234d658 admin admin] PATH_INFO: `/v3/projects/4253689ef8e74abf938b065e25e14c35/users/f27905cbeeef43919eef07848dd8b8ea/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:06.975795 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a902a7c-5f56-4c9e-b0b9-bc36a234d658 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4253689ef8e74abf938b065e25e14c35, user_id=f27905cbeeef43919eef07848dd8b8ea, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:06.979138 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a902a7c-5f56-4c9e-b0b9-bc36a234d658 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:06.991241 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1115/2245] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:20:06 2022] PUT /identity/v3/projects/4253689ef8e74abf938b065e25e14c35/users/f27905cbeeef43919eef07848dd8b8ea/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:20:06.995228 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b34dac-3c84-4c30-9c14-3e2fb2f35cf7 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:07.005118 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b34dac-3c84-4c30-9c14-3e2fb2f35cf7 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:07.060216 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4517dcf7-9728-48ca-b24c-cb61361df335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:07.060809 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4517dcf7-9728-48ca-b24c-cb61361df335 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:07.061090 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4517dcf7-9728-48ca-b24c-cb61361df335 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:07.061366 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4517dcf7-9728-48ca-b24c-cb61361df335 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:07.062060 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4517dcf7-9728-48ca-b24c-cb61361df335 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:07.063639 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4517dcf7-9728-48ca-b24c-cb61361df335 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:07.073591 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1131/2246] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:20:06 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:20:07.077706 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a902a7c-5f56-4c9e-b0b9-bc36a234d658 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:07.087810 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a902a7c-5f56-4c9e-b0b9-bc36a234d658 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:07.092380 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99afa7f8-f2af-4872-860d-93eb846897f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:07.093022 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99afa7f8-f2af-4872-860d-93eb846897f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:07.093305 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99afa7f8-f2af-4872-860d-93eb846897f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:07.093583 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99afa7f8-f2af-4872-860d-93eb846897f5 admin admin] PATH_INFO: `/v3/projects/4253689ef8e74abf938b065e25e14c35/users/f27905cbeeef43919eef07848dd8b8ea/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:07.095505 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99afa7f8-f2af-4872-860d-93eb846897f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4253689ef8e74abf938b065e25e14c35, user_id=f27905cbeeef43919eef07848dd8b8ea, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:07.098161 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99afa7f8-f2af-4872-860d-93eb846897f5 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:07.108850 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1116/2247] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:20:07 2022] PUT /identity/v3/projects/4253689ef8e74abf938b065e25e14c35/users/f27905cbeeef43919eef07848dd8b8ea/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:20:07.120318 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b16f552-f52c-423f-b1f6-57b7a2fe7454 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:07.120791 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b16f552-f52c-423f-b1f6-57b7a2fe7454 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:07.120791 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b16f552-f52c-423f-b1f6-57b7a2fe7454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:07.139879 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:20:07.149491 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-6b16f552-f52c-423f-b1f6-57b7a2fe7454 None None] MFA Rules not processed for user `f27905cbeeef43919eef07848dd8b8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:20:07.150782 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-6b16f552-f52c-423f-b1f6-57b7a2fe7454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:20:07.217542 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1132/2248] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:20:07 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:20:07.228828 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99afa7f8-f2af-4872-860d-93eb846897f5 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:07.230560 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-99afa7f8-f2af-4872-860d-93eb846897f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:20:07.287339 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99afa7f8-f2af-4872-860d-93eb846897f5 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:07.291530 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13035f8-77b5-4b49-9139-4a7abccdedb2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:07.292159 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13035f8-77b5-4b49-9139-4a7abccdedb2 service cinder] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:07.292435 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13035f8-77b5-4b49-9139-4a7abccdedb2 service cinder] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:07.292705 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13035f8-77b5-4b49-9139-4a7abccdedb2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:07.302304 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13035f8-77b5-4b49-9139-4a7abccdedb2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:07.303725 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13035f8-77b5-4b49-9139-4a7abccdedb2 service cinder] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:07.315065 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1117/2249] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:20:07 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:20:07.445911 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b16f552-f52c-423f-b1f6-57b7a2fe7454 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:07.508065 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b16f552-f52c-423f-b1f6-57b7a2fe7454 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:07.513203 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f14de3-c1f5-4739-b18e-2a4d665eabf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:07.513856 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f14de3-c1f5-4739-b18e-2a4d665eabf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:07.514134 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f14de3-c1f5-4739-b18e-2a4d665eabf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:07.514406 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f14de3-c1f5-4739-b18e-2a4d665eabf7 admin admin] PATH_INFO: `/v3/users/88efaa7a600c418fa7df9f8efd3a5ba3` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:07.515522 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f14de3-c1f5-4739-b18e-2a4d665eabf7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88efaa7a600c418fa7df9f8efd3a5ba3)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:07.517423 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f14de3-c1f5-4739-b18e-2a4d665eabf7 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:07.565088 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d8f14de3-c1f5-4739-b18e-2a4d665eabf7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88efaa7a600c418fa7df9f8efd3a5ba3'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:07.565764 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d8f14de3-c1f5-4739-b18e-2a4d665eabf7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88efaa7a600c418fa7df9f8efd3a5ba3'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:07.573104 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d8f14de3-c1f5-4739-b18e-2a4d665eabf7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88efaa7a600c418fa7df9f8efd3a5ba3'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:07.584921 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d8f14de3-c1f5-4739-b18e-2a4d665eabf7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88efaa7a600c418fa7df9f8efd3a5ba3'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:07.591852 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d8f14de3-c1f5-4739-b18e-2a4d665eabf7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88efaa7a600c418fa7df9f8efd3a5ba3'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:07.597766 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1133/2250] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 09:20:07 2022] DELETE /identity/v3/users/88efaa7a600c418fa7df9f8efd3a5ba3 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:20:07.602179 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57d2d6-3ff2-4a9f-89e8-e81fba63c4ea None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:07.602470 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57d2d6-3ff2-4a9f-89e8-e81fba63c4ea None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:07.602772 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57d2d6-3ff2-4a9f-89e8-e81fba63c4ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:07.621667 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-8a57d2d6-3ff2-4a9f-89e8-e81fba63c4ea None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:20:07.622835 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-8a57d2d6-3ff2-4a9f-89e8-e81fba63c4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:20:07.684263 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1118/2251] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:20:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:20:07.693841 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f14de3-c1f5-4739-b18e-2a4d665eabf7 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:07.695222 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-d8f14de3-c1f5-4739-b18e-2a4d665eabf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:20:07.709833 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:20:07.753266 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f14de3-c1f5-4739-b18e-2a4d665eabf7 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:07.756931 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf1e188-7b8b-4e3d-a5c6-a67fd7c10181 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:07.757536 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf1e188-7b8b-4e3d-a5c6-a67fd7c10181 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:07.757846 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf1e188-7b8b-4e3d-a5c6-a67fd7c10181 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:07.758126 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf1e188-7b8b-4e3d-a5c6-a67fd7c10181 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:07.767074 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf1e188-7b8b-4e3d-a5c6-a67fd7c10181 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:07.768493 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf1e188-7b8b-4e3d-a5c6-a67fd7c10181 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:07.778332 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1134/2252] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:20:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 09:20:08.042689 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a57d2d6-3ff2-4a9f-89e8-e81fba63c4ea None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:08.044269 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-8a57d2d6-3ff2-4a9f-89e8-e81fba63c4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:20:08.062152 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a57d2d6-3ff2-4a9f-89e8-e81fba63c4ea None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:08.067593 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bf8e0f-8f17-425c-a260-06d47bedeb15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:08.068296 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bf8e0f-8f17-425c-a260-06d47bedeb15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:08.068486 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bf8e0f-8f17-425c-a260-06d47bedeb15 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:08.068759 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bf8e0f-8f17-425c-a260-06d47bedeb15 admin admin] PATH_INFO: `/v3/projects/af3a4a0709904160be303f5093d8ba66` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:08.069897 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bf8e0f-8f17-425c-a260-06d47bedeb15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af3a4a0709904160be303f5093d8ba66)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:08.071794 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bf8e0f-8f17-425c-a260-06d47bedeb15 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:08.117159 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-34bf8e0f-8f17-425c-a260-06d47bedeb15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af3a4a0709904160be303f5093d8ba66', 'request_id': 'req-34bf8e0f-8f17-425c-a260-06d47bedeb15'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:08.124018 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-34bf8e0f-8f17-425c-a260-06d47bedeb15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af3a4a0709904160be303f5093d8ba66', 'request_id': 'req-34bf8e0f-8f17-425c-a260-06d47bedeb15'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:08.131194 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-34bf8e0f-8f17-425c-a260-06d47bedeb15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af3a4a0709904160be303f5093d8ba66', 'request_id': 'req-34bf8e0f-8f17-425c-a260-06d47bedeb15'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:08.135311 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:20:08.143895 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-34bf8e0f-8f17-425c-a260-06d47bedeb15 admin admin] The token cache is being invalidate because project af3a4a0709904160be303f5093d8ba66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 09:20:08.144400 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-34bf8e0f-8f17-425c-a260-06d47bedeb15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:08.145322 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-34bf8e0f-8f17-425c-a260-06d47bedeb15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:08.146824 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1119/2253] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 09:20:08 2022] DELETE /identity/v3/projects/af3a4a0709904160be303f5093d8ba66 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:20:08.150803 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf1e188-7b8b-4e3d-a5c6-a67fd7c10181 service neutron] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:08.160398 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf1e188-7b8b-4e3d-a5c6-a67fd7c10181 service neutron] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:08.216007 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d4d6e1-41f6-4c50-a326-b8b33d83ad94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:08.216623 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d4d6e1-41f6-4c50-a326-b8b33d83ad94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:08.216903 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d4d6e1-41f6-4c50-a326-b8b33d83ad94 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:08.217279 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d4d6e1-41f6-4c50-a326-b8b33d83ad94 admin admin] PATH_INFO: `/v3/users/f27905cbeeef43919eef07848dd8b8ea` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:08.218412 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d4d6e1-41f6-4c50-a326-b8b33d83ad94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f27905cbeeef43919eef07848dd8b8ea)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:08.220307 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d4d6e1-41f6-4c50-a326-b8b33d83ad94 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:08.265334 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-12d4d6e1-41f6-4c50-a326-b8b33d83ad94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f27905cbeeef43919eef07848dd8b8ea'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:08.266027 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-12d4d6e1-41f6-4c50-a326-b8b33d83ad94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f27905cbeeef43919eef07848dd8b8ea'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:08.272797 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-12d4d6e1-41f6-4c50-a326-b8b33d83ad94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f27905cbeeef43919eef07848dd8b8ea'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:08.285105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-12d4d6e1-41f6-4c50-a326-b8b33d83ad94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f27905cbeeef43919eef07848dd8b8ea'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:08.292615 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-12d4d6e1-41f6-4c50-a326-b8b33d83ad94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f27905cbeeef43919eef07848dd8b8ea'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:08.299501 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1135/2254] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 09:20:08 2022] DELETE /identity/v3/users/f27905cbeeef43919eef07848dd8b8ea => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:20:08.571164 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bf8e0f-8f17-425c-a260-06d47bedeb15 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:08.573240 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-34bf8e0f-8f17-425c-a260-06d47bedeb15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:20:08.596286 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bf8e0f-8f17-425c-a260-06d47bedeb15 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:08.661892 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d147f6cd-4764-4e22-ba55-b191fbfcb287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:08.662531 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d147f6cd-4764-4e22-ba55-b191fbfcb287 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:08.662813 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d147f6cd-4764-4e22-ba55-b191fbfcb287 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:08.663086 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d147f6cd-4764-4e22-ba55-b191fbfcb287 admin admin] PATH_INFO: `/v3/projects/4253689ef8e74abf938b065e25e14c35` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:08.664175 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d147f6cd-4764-4e22-ba55-b191fbfcb287 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4253689ef8e74abf938b065e25e14c35)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:08.666734 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d147f6cd-4764-4e22-ba55-b191fbfcb287 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:08.714092 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-d147f6cd-4764-4e22-ba55-b191fbfcb287 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4253689ef8e74abf938b065e25e14c35', 'request_id': 'req-d147f6cd-4764-4e22-ba55-b191fbfcb287'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:08.720265 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-d147f6cd-4764-4e22-ba55-b191fbfcb287 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4253689ef8e74abf938b065e25e14c35', 'request_id': 'req-d147f6cd-4764-4e22-ba55-b191fbfcb287'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:08.726653 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-d147f6cd-4764-4e22-ba55-b191fbfcb287 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4253689ef8e74abf938b065e25e14c35', 'request_id': 'req-d147f6cd-4764-4e22-ba55-b191fbfcb287'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:08.736784 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-d147f6cd-4764-4e22-ba55-b191fbfcb287 admin admin] The token cache is being invalidate because project 4253689ef8e74abf938b065e25e14c35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 09:20:08.737067 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-d147f6cd-4764-4e22-ba55-b191fbfcb287 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:08.737693 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-d147f6cd-4764-4e22-ba55-b191fbfcb287 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:08.739140 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1120/2255] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 09:20:08 2022] DELETE /identity/v3/projects/4253689ef8e74abf938b065e25e14c35 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:20:08.744036 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9faffa-fd48-4050-804a-934c35f652d0 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:08.744325 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9faffa-fd48-4050-804a-934c35f652d0 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:08.744603 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9faffa-fd48-4050-804a-934c35f652d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:08.764287 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-de9faffa-fd48-4050-804a-934c35f652d0 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:20:08.765481 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-de9faffa-fd48-4050-804a-934c35f652d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:20:08.795255 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:20:08.825408 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1136/2256] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:20:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:20:08.832991 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1460b41-5db6-43f3-a655-6135a7142e1c None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:08.833275 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1460b41-5db6-43f3-a655-6135a7142e1c None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:08.833581 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1460b41-5db6-43f3-a655-6135a7142e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:08.853366 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-e1460b41-5db6-43f3-a655-6135a7142e1c None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:20:08.854579 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-e1460b41-5db6-43f3-a655-6135a7142e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:20:08.862261 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1121/2257] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:20:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:20:08.866317 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9faffa-fd48-4050-804a-934c35f652d0 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:08.875813 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9faffa-fd48-4050-804a-934c35f652d0 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:08.880371 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c170994-eafa-4c99-9e06-d604a23c6758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:08.880956 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c170994-eafa-4c99-9e06-d604a23c6758 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:08.881242 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c170994-eafa-4c99-9e06-d604a23c6758 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:08.881521 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c170994-eafa-4c99-9e06-d604a23c6758 admin admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:08.882197 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c170994-eafa-4c99-9e06-d604a23c6758 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 09:20:08.882503 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c170994-eafa-4c99-9e06-d604a23c6758 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:08.884092 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c170994-eafa-4c99-9e06-d604a23c6758 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:08.896046 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1137/2258] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 09:20:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 09:20:08.900644 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1460b41-5db6-43f3-a655-6135a7142e1c None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:08.909729 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1460b41-5db6-43f3-a655-6135a7142e1c None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:08.914457 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8860cee8-1c79-48a4-8912-cb7a0716ec53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:08.914987 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8860cee8-1c79-48a4-8912-cb7a0716ec53 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:08.915264 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8860cee8-1c79-48a4-8912-cb7a0716ec53 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:08.915533 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8860cee8-1c79-48a4-8912-cb7a0716ec53 admin admin] PATH_INFO: `/v3/projects` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:08.916164 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8860cee8-1c79-48a4-8912-cb7a0716ec53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:08.918087 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8860cee8-1c79-48a4-8912-cb7a0716ec53 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:08.944689 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1122/2259] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 09:20:08 2022] POST /identity/v3/projects => generated 387 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:20:08.949217 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c170994-eafa-4c99-9e06-d604a23c6758 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:08.959035 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c170994-eafa-4c99-9e06-d604a23c6758 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:09.016528 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae890e-5568-4a5c-a6eb-2fc9b488e226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:09.017140 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae890e-5568-4a5c-a6eb-2fc9b488e226 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:09.017429 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae890e-5568-4a5c-a6eb-2fc9b488e226 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:09.017793 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae890e-5568-4a5c-a6eb-2fc9b488e226 admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:09.018429 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ae890e-5568-4a5c-a6eb-2fc9b488e226 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:09.020342 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ae890e-5568-4a5c-a6eb-2fc9b488e226 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:09.043437 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1138/2260] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 09:20:08 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 09:20:09.047482 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8860cee8-1c79-48a4-8912-cb7a0716ec53 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:09.057734 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8860cee8-1c79-48a4-8912-cb7a0716ec53 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:09.062195 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b42f6d-b851-4f7a-baf8-80b3e041c891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:09.062783 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b42f6d-b851-4f7a-baf8-80b3e041c891 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:09.063060 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b42f6d-b851-4f7a-baf8-80b3e041c891 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:09.063333 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b42f6d-b851-4f7a-baf8-80b3e041c891 admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:09.063994 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b42f6d-b851-4f7a-baf8-80b3e041c891 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:09.065608 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b42f6d-b851-4f7a-baf8-80b3e041c891 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:09.075321 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1123/2261] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:20:09 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:20:09.078951 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae890e-5568-4a5c-a6eb-2fc9b488e226 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:09.089442 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae890e-5568-4a5c-a6eb-2fc9b488e226 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:09.094086 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0601a6-2664-4475-8a4b-b5061fadd307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:09.094679 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0601a6-2664-4475-8a4b-b5061fadd307 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:09.094963 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0601a6-2664-4475-8a4b-b5061fadd307 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:09.095244 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0601a6-2664-4475-8a4b-b5061fadd307 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:09.095886 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0601a6-2664-4475-8a4b-b5061fadd307 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:09.097471 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0601a6-2664-4475-8a4b-b5061fadd307 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:09.108541 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1139/2262] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:20:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:20:09.112689 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b42f6d-b851-4f7a-baf8-80b3e041c891 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:09.123078 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b42f6d-b851-4f7a-baf8-80b3e041c891 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:09.127726 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec358db7-4ab1-49ee-9865-da298fe0b20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:09.128376 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec358db7-4ab1-49ee-9865-da298fe0b20f admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:09.128688 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec358db7-4ab1-49ee-9865-da298fe0b20f admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:09.128968 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec358db7-4ab1-49ee-9865-da298fe0b20f admin admin] PATH_INFO: `/v3/projects/aa4e88db184744faad4ad9508d90a3f2/users/1c8805b3cff34ee8a5d27b4ebf05889c/roles/3248f0268738489b883041851d681bb7` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:09.143271 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec358db7-4ab1-49ee-9865-da298fe0b20f admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa4e88db184744faad4ad9508d90a3f2, user_id=1c8805b3cff34ee8a5d27b4ebf05889c, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:09.146018 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec358db7-4ab1-49ee-9865-da298fe0b20f admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:09.159222 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1124/2263] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:20:09 2022] PUT /identity/v3/projects/aa4e88db184744faad4ad9508d90a3f2/users/1c8805b3cff34ee8a5d27b4ebf05889c/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 09:20:09.170133 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf2f3a-589a-4268-b54b-34f239920026 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:09.170607 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf2f3a-589a-4268-b54b-34f239920026 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:09.170607 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf2f3a-589a-4268-b54b-34f239920026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:09.197754 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-1aaf2f3a-589a-4268-b54b-34f239920026 None None] MFA Rules not processed for user `1c8805b3cff34ee8a5d27b4ebf05889c`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:20:09.199121 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-1aaf2f3a-589a-4268-b54b-34f239920026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:20:09.263019 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1140/2264] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:20:09 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:20:09.267434 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec358db7-4ab1-49ee-9865-da298fe0b20f admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:09.288250 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:20:09.333325 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec358db7-4ab1-49ee-9865-da298fe0b20f admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:09.338508 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c270f63-22a0-4402-be4d-fa513f92595d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:09.339111 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c270f63-22a0-4402-be4d-fa513f92595d admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:09.339388 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c270f63-22a0-4402-be4d-fa513f92595d admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:09.339658 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c270f63-22a0-4402-be4d-fa513f92595d admin admin] PATH_INFO: `/v3/projects` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:09.340321 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c270f63-22a0-4402-be4d-fa513f92595d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:09.342286 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c270f63-22a0-4402-be4d-fa513f92595d admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:09.368068 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1125/2265] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 09:20:09 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 09:20:09.372674 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaf2f3a-589a-4268-b54b-34f239920026 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:09.383172 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaf2f3a-589a-4268-b54b-34f239920026 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:09.411598 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:20:09.441238 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1c518c-da95-4ac4-8158-54bd986a49e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:09.441828 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1c518c-da95-4ac4-8158-54bd986a49e8 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:09.442108 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1c518c-da95-4ac4-8158-54bd986a49e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:09.442377 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1c518c-da95-4ac4-8158-54bd986a49e8 admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:09.442995 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1c518c-da95-4ac4-8158-54bd986a49e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:09.444883 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1c518c-da95-4ac4-8158-54bd986a49e8 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:09.465293 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1141/2266] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 09:20:09 2022] POST /identity/v3/users => generated 299 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:20:09.469313 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c270f63-22a0-4402-be4d-fa513f92595d admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:09.479898 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c270f63-22a0-4402-be4d-fa513f92595d admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:09.484375 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c939490b-2ccf-4d23-b6ad-f3a8f2757e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:09.484966 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c939490b-2ccf-4d23-b6ad-f3a8f2757e12 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:09.485243 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c939490b-2ccf-4d23-b6ad-f3a8f2757e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:09.485513 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c939490b-2ccf-4d23-b6ad-f3a8f2757e12 admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:09.486207 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c939490b-2ccf-4d23-b6ad-f3a8f2757e12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:09.487798 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c939490b-2ccf-4d23-b6ad-f3a8f2757e12 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:09.498765 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1126/2267] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:20:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:20:09.502813 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1c518c-da95-4ac4-8158-54bd986a49e8 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:09.512728 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1c518c-da95-4ac4-8158-54bd986a49e8 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:09.517297 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b056cb39-fc6e-47c7-843b-39a2f9b73014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:09.517964 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b056cb39-fc6e-47c7-843b-39a2f9b73014 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:09.518247 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b056cb39-fc6e-47c7-843b-39a2f9b73014 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:09.518524 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b056cb39-fc6e-47c7-843b-39a2f9b73014 admin admin] PATH_INFO: `/v3/projects/b930ea5f970a40ce9279af1c98aa35a4/users/667d0ddc24dc4a488579aa665b7c3cf3/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:09.533288 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b056cb39-fc6e-47c7-843b-39a2f9b73014 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b930ea5f970a40ce9279af1c98aa35a4, user_id=667d0ddc24dc4a488579aa665b7c3cf3, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:09.536012 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b056cb39-fc6e-47c7-843b-39a2f9b73014 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:09.549370 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1142/2268] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:20:09 2022] PUT /identity/v3/projects/b930ea5f970a40ce9279af1c98aa35a4/users/667d0ddc24dc4a488579aa665b7c3cf3/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:20:09.553335 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c939490b-2ccf-4d23-b6ad-f3a8f2757e12 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:09.563791 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c939490b-2ccf-4d23-b6ad-f3a8f2757e12 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:09.618674 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db2e488-e962-4fbb-bd4f-60f78998646f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:09.619303 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db2e488-e962-4fbb-bd4f-60f78998646f admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:09.619583 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db2e488-e962-4fbb-bd4f-60f78998646f admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:09.619862 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db2e488-e962-4fbb-bd4f-60f78998646f admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:09.620516 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db2e488-e962-4fbb-bd4f-60f78998646f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:09.622145 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db2e488-e962-4fbb-bd4f-60f78998646f admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:09.635528 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1127/2269] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:20:09 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:20:09.637144 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b056cb39-fc6e-47c7-843b-39a2f9b73014 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:09.647210 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b056cb39-fc6e-47c7-843b-39a2f9b73014 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:09.652426 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465b721c-cd9a-4043-8813-037004fe5a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:09.653084 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b721c-cd9a-4043-8813-037004fe5a13 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:09.653393 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b721c-cd9a-4043-8813-037004fe5a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:09.653698 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b721c-cd9a-4043-8813-037004fe5a13 admin admin] PATH_INFO: `/v3/projects/b930ea5f970a40ce9279af1c98aa35a4/users/667d0ddc24dc4a488579aa665b7c3cf3/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:09.655634 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465b721c-cd9a-4043-8813-037004fe5a13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b930ea5f970a40ce9279af1c98aa35a4, user_id=667d0ddc24dc4a488579aa665b7c3cf3, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:09.658376 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465b721c-cd9a-4043-8813-037004fe5a13 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:09.675243 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1143/2270] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:20:09 2022] PUT /identity/v3/projects/b930ea5f970a40ce9279af1c98aa35a4/users/667d0ddc24dc4a488579aa665b7c3cf3/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:20:09.690694 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6dc15-37a9-4b83-a183-df8f65f927b6 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:09.691548 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6dc15-37a9-4b83-a183-df8f65f927b6 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:09.691769 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6dc15-37a9-4b83-a183-df8f65f927b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:09.719986 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-d3a6dc15-37a9-4b83-a183-df8f65f927b6 None None] MFA Rules not processed for user `667d0ddc24dc4a488579aa665b7c3cf3`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:20:09.721192 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-d3a6dc15-37a9-4b83-a183-df8f65f927b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:20:09.788154 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1128/2271] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:20:09 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:20:09.799145 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465b721c-cd9a-4043-8813-037004fe5a13 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:09.800856 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-465b721c-cd9a-4043-8813-037004fe5a13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:20:09.857266 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465b721c-cd9a-4043-8813-037004fe5a13 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:09.861016 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992015bc-db64-40ee-81a1-729950205827 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:09.861607 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992015bc-db64-40ee-81a1-729950205827 service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:09.861982 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992015bc-db64-40ee-81a1-729950205827 service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:09.862264 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992015bc-db64-40ee-81a1-729950205827 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:09.871665 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992015bc-db64-40ee-81a1-729950205827 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:09.873088 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992015bc-db64-40ee-81a1-729950205827 service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:09.882293 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1144/2272] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:20:09 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:20:10.416548 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a6dc15-37a9-4b83-a183-df8f65f927b6 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:10.434100 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:20:10.479191 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a6dc15-37a9-4b83-a183-df8f65f927b6 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:10.483633 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecebbda-b7d6-4d22-99e9-fbcf8a0a18cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:10.484260 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecebbda-b7d6-4d22-99e9-fbcf8a0a18cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:10.484542 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecebbda-b7d6-4d22-99e9-fbcf8a0a18cc admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:10.484834 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecebbda-b7d6-4d22-99e9-fbcf8a0a18cc admin admin] PATH_INFO: `/v3/users/1c8805b3cff34ee8a5d27b4ebf05889c` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:10.485957 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecebbda-b7d6-4d22-99e9-fbcf8a0a18cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c8805b3cff34ee8a5d27b4ebf05889c)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:10.487883 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecebbda-b7d6-4d22-99e9-fbcf8a0a18cc admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:10.532939 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-8ecebbda-b7d6-4d22-99e9-fbcf8a0a18cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c8805b3cff34ee8a5d27b4ebf05889c'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:10.533587 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-8ecebbda-b7d6-4d22-99e9-fbcf8a0a18cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c8805b3cff34ee8a5d27b4ebf05889c'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:10.545842 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-8ecebbda-b7d6-4d22-99e9-fbcf8a0a18cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c8805b3cff34ee8a5d27b4ebf05889c'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:10.553286 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-8ecebbda-b7d6-4d22-99e9-fbcf8a0a18cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c8805b3cff34ee8a5d27b4ebf05889c'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:10.553975 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-8ecebbda-b7d6-4d22-99e9-fbcf8a0a18cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c8805b3cff34ee8a5d27b4ebf05889c'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:10.566575 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1129/2273] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 09:20:10 2022] DELETE /identity/v3/users/1c8805b3cff34ee8a5d27b4ebf05889c => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:20:10.571023 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5fc2d-d62f-474f-8671-c8952f8b115e None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:10.571477 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5fc2d-d62f-474f-8671-c8952f8b115e None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:10.571590 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5fc2d-d62f-474f-8671-c8952f8b115e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:10.590847 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-1ac5fc2d-d62f-474f-8671-c8952f8b115e None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:20:10.592053 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-1ac5fc2d-d62f-474f-8671-c8952f8b115e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:20:10.651121 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1145/2274] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:20:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:20:10.660730 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecebbda-b7d6-4d22-99e9-fbcf8a0a18cc admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:10.662170 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-8ecebbda-b7d6-4d22-99e9-fbcf8a0a18cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:20:10.721167 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecebbda-b7d6-4d22-99e9-fbcf8a0a18cc admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:10.724807 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1793722a-e43d-42ca-a1fc-a90cdbb14d8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:10.725413 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1793722a-e43d-42ca-a1fc-a90cdbb14d8e service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:10.725722 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1793722a-e43d-42ca-a1fc-a90cdbb14d8e service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:10.726005 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1793722a-e43d-42ca-a1fc-a90cdbb14d8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:10.734930 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1793722a-e43d-42ca-a1fc-a90cdbb14d8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:10.736293 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1793722a-e43d-42ca-a1fc-a90cdbb14d8e service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:10.746187 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1130/2275] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:20:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:20:11.002440 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac5fc2d-d62f-474f-8671-c8952f8b115e None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:11.003880 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-1ac5fc2d-d62f-474f-8671-c8952f8b115e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:20:11.016848 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac5fc2d-d62f-474f-8671-c8952f8b115e None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:11.021426 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531d0f78-244b-4eb5-a0e9-90de3dcec193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:11.022104 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d0f78-244b-4eb5-a0e9-90de3dcec193 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:11.022387 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d0f78-244b-4eb5-a0e9-90de3dcec193 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:11.022662 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d0f78-244b-4eb5-a0e9-90de3dcec193 admin admin] PATH_INFO: `/v3/projects/aa4e88db184744faad4ad9508d90a3f2` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:11.023795 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531d0f78-244b-4eb5-a0e9-90de3dcec193 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa4e88db184744faad4ad9508d90a3f2)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:11.026465 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531d0f78-244b-4eb5-a0e9-90de3dcec193 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:11.049482 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:20:11.074241 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-531d0f78-244b-4eb5-a0e9-90de3dcec193 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa4e88db184744faad4ad9508d90a3f2', 'request_id': 'req-531d0f78-244b-4eb5-a0e9-90de3dcec193'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:11.080339 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-531d0f78-244b-4eb5-a0e9-90de3dcec193 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa4e88db184744faad4ad9508d90a3f2', 'request_id': 'req-531d0f78-244b-4eb5-a0e9-90de3dcec193'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:11.088205 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-531d0f78-244b-4eb5-a0e9-90de3dcec193 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa4e88db184744faad4ad9508d90a3f2', 'request_id': 'req-531d0f78-244b-4eb5-a0e9-90de3dcec193'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:11.098508 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-531d0f78-244b-4eb5-a0e9-90de3dcec193 admin admin] The token cache is being invalidate because project aa4e88db184744faad4ad9508d90a3f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 09:20:11.098901 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-531d0f78-244b-4eb5-a0e9-90de3dcec193 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:11.099545 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-531d0f78-244b-4eb5-a0e9-90de3dcec193 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:11.101017 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1146/2276] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 09:20:11 2022] DELETE /identity/v3/projects/aa4e88db184744faad4ad9508d90a3f2 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:20:11.105379 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1793722a-e43d-42ca-a1fc-a90cdbb14d8e service neutron] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:11.114887 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1793722a-e43d-42ca-a1fc-a90cdbb14d8e service neutron] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:11.148639 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:20:11.173957 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099b911b-574b-4e40-93aa-a87b31c11c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:11.174583 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b911b-574b-4e40-93aa-a87b31c11c29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:11.174892 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b911b-574b-4e40-93aa-a87b31c11c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:11.175169 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b911b-574b-4e40-93aa-a87b31c11c29 admin admin] PATH_INFO: `/v3/users/667d0ddc24dc4a488579aa665b7c3cf3` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:11.176278 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099b911b-574b-4e40-93aa-a87b31c11c29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=667d0ddc24dc4a488579aa665b7c3cf3)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:11.178220 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099b911b-574b-4e40-93aa-a87b31c11c29 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:11.224265 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-099b911b-574b-4e40-93aa-a87b31c11c29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '667d0ddc24dc4a488579aa665b7c3cf3'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:11.224905 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-099b911b-574b-4e40-93aa-a87b31c11c29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '667d0ddc24dc4a488579aa665b7c3cf3'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:11.237222 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-099b911b-574b-4e40-93aa-a87b31c11c29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '667d0ddc24dc4a488579aa665b7c3cf3'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:11.244413 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-099b911b-574b-4e40-93aa-a87b31c11c29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '667d0ddc24dc4a488579aa665b7c3cf3'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:11.245081 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-099b911b-574b-4e40-93aa-a87b31c11c29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '667d0ddc24dc4a488579aa665b7c3cf3'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:11.256884 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1131/2277] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 09:20:11 2022] DELETE /identity/v3/users/667d0ddc24dc4a488579aa665b7c3cf3 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:20:11.530456 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531d0f78-244b-4eb5-a0e9-90de3dcec193 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:11.531950 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-531d0f78-244b-4eb5-a0e9-90de3dcec193 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:20:11.545035 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531d0f78-244b-4eb5-a0e9-90de3dcec193 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:11.602505 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bb29eb-4c48-47ec-95bb-df813d3764d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:11.603666 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bb29eb-4c48-47ec-95bb-df813d3764d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:11.603895 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bb29eb-4c48-47ec-95bb-df813d3764d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:11.604167 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bb29eb-4c48-47ec-95bb-df813d3764d6 admin admin] PATH_INFO: `/v3/projects/b930ea5f970a40ce9279af1c98aa35a4` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:11.605267 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bb29eb-4c48-47ec-95bb-df813d3764d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b930ea5f970a40ce9279af1c98aa35a4)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:11.607189 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bb29eb-4c48-47ec-95bb-df813d3764d6 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:11.652396 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-e7bb29eb-4c48-47ec-95bb-df813d3764d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b930ea5f970a40ce9279af1c98aa35a4', 'request_id': 'req-e7bb29eb-4c48-47ec-95bb-df813d3764d6'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:11.658540 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-e7bb29eb-4c48-47ec-95bb-df813d3764d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b930ea5f970a40ce9279af1c98aa35a4', 'request_id': 'req-e7bb29eb-4c48-47ec-95bb-df813d3764d6'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:11.666807 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-e7bb29eb-4c48-47ec-95bb-df813d3764d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b930ea5f970a40ce9279af1c98aa35a4', 'request_id': 'req-e7bb29eb-4c48-47ec-95bb-df813d3764d6'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:11.677629 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-e7bb29eb-4c48-47ec-95bb-df813d3764d6 admin admin] The token cache is being invalidate because project b930ea5f970a40ce9279af1c98aa35a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 09:20:11.677945 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-e7bb29eb-4c48-47ec-95bb-df813d3764d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:11.678570 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-e7bb29eb-4c48-47ec-95bb-df813d3764d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:11.680043 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1147/2278] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 09:20:11 2022] DELETE /identity/v3/projects/b930ea5f970a40ce9279af1c98aa35a4 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:20:11.685450 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ce3d03-5ad0-4f38-b6aa-4d439900f358 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:11.685949 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ce3d03-5ad0-4f38-b6aa-4d439900f358 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:11.686040 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ce3d03-5ad0-4f38-b6aa-4d439900f358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:11.705610 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-e9ce3d03-5ad0-4f38-b6aa-4d439900f358 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:20:11.706915 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-e9ce3d03-5ad0-4f38-b6aa-4d439900f358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:20:11.767972 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1132/2279] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:20:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:20:11.776187 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef121486-55ba-4be2-9cac-015983b5b002 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:11.776424 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef121486-55ba-4be2-9cac-015983b5b002 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:11.776701 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef121486-55ba-4be2-9cac-015983b5b002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:11.796083 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-ef121486-55ba-4be2-9cac-015983b5b002 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:20:11.797292 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-ef121486-55ba-4be2-9cac-015983b5b002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:20:11.805390 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1148/2280] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:20:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:20:11.809483 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ce3d03-5ad0-4f38-b6aa-4d439900f358 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:11.819148 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ce3d03-5ad0-4f38-b6aa-4d439900f358 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:11.824143 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c06862b-08bf-4288-9ce8-82e438910190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:11.824731 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06862b-08bf-4288-9ce8-82e438910190 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:11.825011 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06862b-08bf-4288-9ce8-82e438910190 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:11.825281 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06862b-08bf-4288-9ce8-82e438910190 admin admin] PATH_INFO: `/v3/domains` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:11.825942 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c06862b-08bf-4288-9ce8-82e438910190 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 09:20:11.826244 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c06862b-08bf-4288-9ce8-82e438910190 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:11.827829 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c06862b-08bf-4288-9ce8-82e438910190 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:11.839576 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1133/2281] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 09:20:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 09:20:11.843579 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef121486-55ba-4be2-9cac-015983b5b002 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:11.852593 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef121486-55ba-4be2-9cac-015983b5b002 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:11.857894 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57105bfe-f0ee-43ad-b4fa-81acc0ff1390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:11.858490 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57105bfe-f0ee-43ad-b4fa-81acc0ff1390 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:11.858774 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57105bfe-f0ee-43ad-b4fa-81acc0ff1390 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:11.859048 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57105bfe-f0ee-43ad-b4fa-81acc0ff1390 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:11.859702 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57105bfe-f0ee-43ad-b4fa-81acc0ff1390 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:11.861584 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57105bfe-f0ee-43ad-b4fa-81acc0ff1390 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:11.888407 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1149/2282] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 09:20:11 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:20:11.892612 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c06862b-08bf-4288-9ce8-82e438910190 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:11.902527 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c06862b-08bf-4288-9ce8-82e438910190 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:11.945206 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:20:11.961488 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f038af08-d082-41bf-be4a-c69ef371b5fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:11.962125 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f038af08-d082-41bf-be4a-c69ef371b5fe admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:11.962409 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f038af08-d082-41bf-be4a-c69ef371b5fe admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:11.963185 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f038af08-d082-41bf-be4a-c69ef371b5fe admin admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:11.963959 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f038af08-d082-41bf-be4a-c69ef371b5fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:11.965900 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f038af08-d082-41bf-be4a-c69ef371b5fe admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:11.987081 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1134/2283] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 09:20:11 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:20:11.991164 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57105bfe-f0ee-43ad-b4fa-81acc0ff1390 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:12.001450 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57105bfe-f0ee-43ad-b4fa-81acc0ff1390 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:12.006093 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a122d74-a8d1-4773-a717-ce01c2ebd8de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:12.006690 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a122d74-a8d1-4773-a717-ce01c2ebd8de admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:12.006974 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a122d74-a8d1-4773-a717-ce01c2ebd8de admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:12.007250 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a122d74-a8d1-4773-a717-ce01c2ebd8de admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:12.007908 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a122d74-a8d1-4773-a717-ce01c2ebd8de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:12.009548 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a122d74-a8d1-4773-a717-ce01c2ebd8de admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:12.014691 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:20:12.019837 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1150/2284] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:20:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:20:12.024310 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f038af08-d082-41bf-be4a-c69ef371b5fe admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:12.035430 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f038af08-d082-41bf-be4a-c69ef371b5fe admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:12.039950 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b43df7-0609-4fbe-bd4c-cf7b4f3a355a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:12.040543 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b43df7-0609-4fbe-bd4c-cf7b4f3a355a admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:12.041454 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b43df7-0609-4fbe-bd4c-cf7b4f3a355a admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:12.041962 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b43df7-0609-4fbe-bd4c-cf7b4f3a355a admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:12.042619 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b43df7-0609-4fbe-bd4c-cf7b4f3a355a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:12.044213 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b43df7-0609-4fbe-bd4c-cf7b4f3a355a admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:12.055527 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1135/2285] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:20:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:20:12.059570 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a122d74-a8d1-4773-a717-ce01c2ebd8de admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:12.069382 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a122d74-a8d1-4773-a717-ce01c2ebd8de admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:12.074202 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f025f10b-4bcb-4674-8f17-c2163eec9aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:12.074855 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f025f10b-4bcb-4674-8f17-c2163eec9aae admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:12.075162 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f025f10b-4bcb-4674-8f17-c2163eec9aae admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:12.075438 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f025f10b-4bcb-4674-8f17-c2163eec9aae admin admin] PATH_INFO: `/v3/projects/ad494b1b1a224c74983167a37132b355/users/ee380dd7dd7e4d968e271b0b45a8a966/roles/3248f0268738489b883041851d681bb7` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:12.090276 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f025f10b-4bcb-4674-8f17-c2163eec9aae admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad494b1b1a224c74983167a37132b355, user_id=ee380dd7dd7e4d968e271b0b45a8a966, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:12.092916 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f025f10b-4bcb-4674-8f17-c2163eec9aae admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:12.106123 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1151/2286] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:20:12 2022] PUT /identity/v3/projects/ad494b1b1a224c74983167a37132b355/users/ee380dd7dd7e4d968e271b0b45a8a966/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:20:12.116365 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f7cd4-14e2-4e73-b133-4cfad410c484 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:12.116685 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f7cd4-14e2-4e73-b133-4cfad410c484 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:12.116965 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f7cd4-14e2-4e73-b133-4cfad410c484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:12.144004 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-b18f7cd4-14e2-4e73-b133-4cfad410c484 None None] MFA Rules not processed for user `ee380dd7dd7e4d968e271b0b45a8a966`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:20:12.145210 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-b18f7cd4-14e2-4e73-b133-4cfad410c484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:20:12.213138 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1136/2287] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:20:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:20:12.214530 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f025f10b-4bcb-4674-8f17-c2163eec9aae admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:12.276593 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f025f10b-4bcb-4674-8f17-c2163eec9aae admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:12.281228 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7109f5-d38c-400f-a23b-29458860cac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:12.281859 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7109f5-d38c-400f-a23b-29458860cac9 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:12.282148 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7109f5-d38c-400f-a23b-29458860cac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:12.282431 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7109f5-d38c-400f-a23b-29458860cac9 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:12.283074 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7109f5-d38c-400f-a23b-29458860cac9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:12.284976 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7109f5-d38c-400f-a23b-29458860cac9 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:12.311355 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1152/2288] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 09:20:12 2022] POST /identity/v3/projects => generated 375 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:20:12.315381 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f7cd4-14e2-4e73-b133-4cfad410c484 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:12.326405 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f7cd4-14e2-4e73-b133-4cfad410c484 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:12.381798 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c542a876-7fdd-4926-8a2a-4caafe35bb57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:12.382408 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542a876-7fdd-4926-8a2a-4caafe35bb57 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:12.382693 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542a876-7fdd-4926-8a2a-4caafe35bb57 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:12.382980 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542a876-7fdd-4926-8a2a-4caafe35bb57 admin admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:12.383626 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c542a876-7fdd-4926-8a2a-4caafe35bb57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:12.385569 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c542a876-7fdd-4926-8a2a-4caafe35bb57 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:12.406305 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1137/2289] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 09:20:12 2022] POST /identity/v3/users => generated 294 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:20:12.410233 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7109f5-d38c-400f-a23b-29458860cac9 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:12.420907 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7109f5-d38c-400f-a23b-29458860cac9 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:12.425715 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a64a06d-5496-4807-a237-0ba52d90b395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:12.426306 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64a06d-5496-4807-a237-0ba52d90b395 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:12.426599 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64a06d-5496-4807-a237-0ba52d90b395 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:12.426859 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64a06d-5496-4807-a237-0ba52d90b395 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:12.427494 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a64a06d-5496-4807-a237-0ba52d90b395 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:12.429066 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a64a06d-5496-4807-a237-0ba52d90b395 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:12.440098 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1153/2290] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:20:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:20:12.443946 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c542a876-7fdd-4926-8a2a-4caafe35bb57 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:12.453808 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c542a876-7fdd-4926-8a2a-4caafe35bb57 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:12.458436 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66acacb-3a49-4654-9912-aad6b11ca55e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:12.459081 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66acacb-3a49-4654-9912-aad6b11ca55e admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:12.459366 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66acacb-3a49-4654-9912-aad6b11ca55e admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:12.459647 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66acacb-3a49-4654-9912-aad6b11ca55e admin admin] PATH_INFO: `/v3/projects/da740db9b3264d6f964d2f26ba586173/users/07f148465ace471e973cafa2e73f9e88/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:12.473996 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66acacb-3a49-4654-9912-aad6b11ca55e admin admin] RBAC: Authorizing `identity:create_grant(project_id=da740db9b3264d6f964d2f26ba586173, user_id=07f148465ace471e973cafa2e73f9e88, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:12.476640 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66acacb-3a49-4654-9912-aad6b11ca55e admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:12.489577 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1138/2291] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:20:12 2022] PUT /identity/v3/projects/da740db9b3264d6f964d2f26ba586173/users/07f148465ace471e973cafa2e73f9e88/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:20:12.493347 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a64a06d-5496-4807-a237-0ba52d90b395 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:12.503840 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a64a06d-5496-4807-a237-0ba52d90b395 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:12.562726 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1098f5-3a41-4dc4-9e66-23140e2bc2cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:12.563258 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1098f5-3a41-4dc4-9e66-23140e2bc2cd admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:12.563540 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1098f5-3a41-4dc4-9e66-23140e2bc2cd admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:12.563813 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1098f5-3a41-4dc4-9e66-23140e2bc2cd admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:12.564446 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1098f5-3a41-4dc4-9e66-23140e2bc2cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:12.566086 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1098f5-3a41-4dc4-9e66-23140e2bc2cd admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:12.575672 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1154/2292] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:20:12 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:20:12.579801 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66acacb-3a49-4654-9912-aad6b11ca55e admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:12.590020 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66acacb-3a49-4654-9912-aad6b11ca55e admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:12.594563 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f91dfb-2c1b-43ef-8bb4-bd04b0781f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:12.595204 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f91dfb-2c1b-43ef-8bb4-bd04b0781f58 admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:12.595486 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f91dfb-2c1b-43ef-8bb4-bd04b0781f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:12.595766 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f91dfb-2c1b-43ef-8bb4-bd04b0781f58 admin admin] PATH_INFO: `/v3/projects/da740db9b3264d6f964d2f26ba586173/users/07f148465ace471e973cafa2e73f9e88/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:12.597650 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f91dfb-2c1b-43ef-8bb4-bd04b0781f58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da740db9b3264d6f964d2f26ba586173, user_id=07f148465ace471e973cafa2e73f9e88, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:12.600295 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f91dfb-2c1b-43ef-8bb4-bd04b0781f58 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:12.611226 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1139/2293] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:20:12 2022] PUT /identity/v3/projects/da740db9b3264d6f964d2f26ba586173/users/07f148465ace471e973cafa2e73f9e88/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:20:12.622944 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b5c8e5-433f-46bd-a907-936113553c82 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:12.623446 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b5c8e5-433f-46bd-a907-936113553c82 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:12.623556 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b5c8e5-433f-46bd-a907-936113553c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:12.650865 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-b9b5c8e5-433f-46bd-a907-936113553c82 None None] MFA Rules not processed for user `07f148465ace471e973cafa2e73f9e88`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:20:12.652063 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-b9b5c8e5-433f-46bd-a907-936113553c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:20:12.687145 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:20:12.724134 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1155/2294] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:20:12 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:20:12.735463 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f91dfb-2c1b-43ef-8bb4-bd04b0781f58 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:12.737264 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-f7f91dfb-2c1b-43ef-8bb4-bd04b0781f58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:20:12.744104 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:20:12.803754 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f91dfb-2c1b-43ef-8bb4-bd04b0781f58 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:12.808233 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4b6240-8b31-465c-9c59-dfda346fdf0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:12.808948 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4b6240-8b31-465c-9c59-dfda346fdf0e service cinder] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:12.809444 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4b6240-8b31-465c-9c59-dfda346fdf0e service cinder] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:12.810093 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4b6240-8b31-465c-9c59-dfda346fdf0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:12.869384 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4b6240-8b31-465c-9c59-dfda346fdf0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:12.870879 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4b6240-8b31-465c-9c59-dfda346fdf0e service cinder] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:12.879955 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1140/2295] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:20:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:20:15.417171 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b5c8e5-433f-46bd-a907-936113553c82 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:15.478561 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b5c8e5-433f-46bd-a907-936113553c82 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:15.483159 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e75e292-ecb2-460e-a546-cb70ea99acd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:15.483787 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e75e292-ecb2-460e-a546-cb70ea99acd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:15.484098 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e75e292-ecb2-460e-a546-cb70ea99acd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:15.484400 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e75e292-ecb2-460e-a546-cb70ea99acd3 admin admin] PATH_INFO: `/v3/users/ee380dd7dd7e4d968e271b0b45a8a966` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:15.485517 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e75e292-ecb2-460e-a546-cb70ea99acd3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee380dd7dd7e4d968e271b0b45a8a966)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:15.487460 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e75e292-ecb2-460e-a546-cb70ea99acd3 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:15.532343 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7e75e292-ecb2-460e-a546-cb70ea99acd3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee380dd7dd7e4d968e271b0b45a8a966'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:15.532975 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7e75e292-ecb2-460e-a546-cb70ea99acd3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee380dd7dd7e4d968e271b0b45a8a966'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:15.539709 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7e75e292-ecb2-460e-a546-cb70ea99acd3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee380dd7dd7e4d968e271b0b45a8a966'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:15.551397 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7e75e292-ecb2-460e-a546-cb70ea99acd3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee380dd7dd7e4d968e271b0b45a8a966'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:15.557924 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7e75e292-ecb2-460e-a546-cb70ea99acd3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee380dd7dd7e4d968e271b0b45a8a966'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:15.563788 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1156/2296] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 09:20:15 2022] DELETE /identity/v3/users/ee380dd7dd7e4d968e271b0b45a8a966 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:20:15.568453 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72953e8e-4421-4a70-bb47-d63348324cd1 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:15.568911 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72953e8e-4421-4a70-bb47-d63348324cd1 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:15.569003 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72953e8e-4421-4a70-bb47-d63348324cd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:15.588838 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-72953e8e-4421-4a70-bb47-d63348324cd1 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:20:15.590064 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-72953e8e-4421-4a70-bb47-d63348324cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:20:15.646406 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1141/2297] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:20:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:20:15.656378 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e75e292-ecb2-460e-a546-cb70ea99acd3 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:15.657820 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-7e75e292-ecb2-460e-a546-cb70ea99acd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:20:15.715759 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e75e292-ecb2-460e-a546-cb70ea99acd3 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:15.719445 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e12f04d-3b8f-40c3-af54-9c7ab271ed08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:15.720052 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12f04d-3b8f-40c3-af54-9c7ab271ed08 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:15.720339 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12f04d-3b8f-40c3-af54-9c7ab271ed08 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:15.720620 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12f04d-3b8f-40c3-af54-9c7ab271ed08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:15.726428 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:20:15.730902 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e12f04d-3b8f-40c3-af54-9c7ab271ed08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:15.732781 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e12f04d-3b8f-40c3-af54-9c7ab271ed08 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:15.742936 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1157/2298] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:20:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:20:16.005335 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72953e8e-4421-4a70-bb47-d63348324cd1 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:16.007090 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-72953e8e-4421-4a70-bb47-d63348324cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:20:16.024258 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72953e8e-4421-4a70-bb47-d63348324cd1 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:16.032028 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ac1078-8730-4779-b805-29bf60002a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:16.033100 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac1078-8730-4779-b805-29bf60002a72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:16.033602 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac1078-8730-4779-b805-29bf60002a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:16.034140 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac1078-8730-4779-b805-29bf60002a72 admin admin] PATH_INFO: `/v3/projects/ad494b1b1a224c74983167a37132b355` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:16.036225 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ac1078-8730-4779-b805-29bf60002a72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad494b1b1a224c74983167a37132b355)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:16.039755 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ac1078-8730-4779-b805-29bf60002a72 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:16.093779 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-45ac1078-8730-4779-b805-29bf60002a72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad494b1b1a224c74983167a37132b355', 'request_id': 'req-45ac1078-8730-4779-b805-29bf60002a72'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:16.101058 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-45ac1078-8730-4779-b805-29bf60002a72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad494b1b1a224c74983167a37132b355', 'request_id': 'req-45ac1078-8730-4779-b805-29bf60002a72'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:16.103538 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:20:16.111107 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-45ac1078-8730-4779-b805-29bf60002a72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad494b1b1a224c74983167a37132b355', 'request_id': 'req-45ac1078-8730-4779-b805-29bf60002a72'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:16.123998 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-45ac1078-8730-4779-b805-29bf60002a72 admin admin] The token cache is being invalidate because project ad494b1b1a224c74983167a37132b355 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 09:20:16.124292 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-45ac1078-8730-4779-b805-29bf60002a72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:16.124956 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-45ac1078-8730-4779-b805-29bf60002a72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:16.126523 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1142/2299] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 09:20:16 2022] DELETE /identity/v3/projects/ad494b1b1a224c74983167a37132b355 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:20:16.130724 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e12f04d-3b8f-40c3-af54-9c7ab271ed08 service neutron] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:16.141000 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e12f04d-3b8f-40c3-af54-9c7ab271ed08 service neutron] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:16.201008 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9c592a-ad90-4ec5-99cf-1f6fc5837ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:16.201562 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9c592a-ad90-4ec5-99cf-1f6fc5837ecc admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:16.201877 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9c592a-ad90-4ec5-99cf-1f6fc5837ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:16.202152 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9c592a-ad90-4ec5-99cf-1f6fc5837ecc admin admin] PATH_INFO: `/v3/users/07f148465ace471e973cafa2e73f9e88` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:16.203420 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9c592a-ad90-4ec5-99cf-1f6fc5837ecc admin admin] RBAC: Authorizing `identity:delete_user(user_id=07f148465ace471e973cafa2e73f9e88)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:16.205319 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9c592a-ad90-4ec5-99cf-1f6fc5837ecc admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:16.249432 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-1a9c592a-ad90-4ec5-99cf-1f6fc5837ecc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07f148465ace471e973cafa2e73f9e88'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:16.250102 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-1a9c592a-ad90-4ec5-99cf-1f6fc5837ecc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07f148465ace471e973cafa2e73f9e88'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:16.256849 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-1a9c592a-ad90-4ec5-99cf-1f6fc5837ecc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07f148465ace471e973cafa2e73f9e88'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:16.270798 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-1a9c592a-ad90-4ec5-99cf-1f6fc5837ecc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07f148465ace471e973cafa2e73f9e88'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:16.279230 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-1a9c592a-ad90-4ec5-99cf-1f6fc5837ecc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07f148465ace471e973cafa2e73f9e88'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:16.285306 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1158/2300] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 09:20:16 2022] DELETE /identity/v3/users/07f148465ace471e973cafa2e73f9e88 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:20:16.552970 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ac1078-8730-4779-b805-29bf60002a72 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:16.554527 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-45ac1078-8730-4779-b805-29bf60002a72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:20:16.572792 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ac1078-8730-4779-b805-29bf60002a72 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:16.635110 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a09dd9-1f38-40e1-a7e8-2c2a7678d88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:16.635757 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a09dd9-1f38-40e1-a7e8-2c2a7678d88b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:16.636037 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a09dd9-1f38-40e1-a7e8-2c2a7678d88b admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:16.636311 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a09dd9-1f38-40e1-a7e8-2c2a7678d88b admin admin] PATH_INFO: `/v3/projects/da740db9b3264d6f964d2f26ba586173` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:16.637427 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a09dd9-1f38-40e1-a7e8-2c2a7678d88b admin admin] RBAC: Authorizing `identity:delete_project(project_id=da740db9b3264d6f964d2f26ba586173)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:16.639455 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a09dd9-1f38-40e1-a7e8-2c2a7678d88b admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:16.687298 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-06a09dd9-1f38-40e1-a7e8-2c2a7678d88b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da740db9b3264d6f964d2f26ba586173', 'request_id': 'req-06a09dd9-1f38-40e1-a7e8-2c2a7678d88b'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:16.693563 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-06a09dd9-1f38-40e1-a7e8-2c2a7678d88b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da740db9b3264d6f964d2f26ba586173', 'request_id': 'req-06a09dd9-1f38-40e1-a7e8-2c2a7678d88b'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:16.702819 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-06a09dd9-1f38-40e1-a7e8-2c2a7678d88b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da740db9b3264d6f964d2f26ba586173', 'request_id': 'req-06a09dd9-1f38-40e1-a7e8-2c2a7678d88b'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:16.714721 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-06a09dd9-1f38-40e1-a7e8-2c2a7678d88b admin admin] The token cache is being invalidate because project da740db9b3264d6f964d2f26ba586173 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 09:20:16.715012 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-06a09dd9-1f38-40e1-a7e8-2c2a7678d88b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:16.715647 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-06a09dd9-1f38-40e1-a7e8-2c2a7678d88b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:16.717125 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1143/2301] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 09:20:16 2022] DELETE /identity/v3/projects/da740db9b3264d6f964d2f26ba586173 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:20:16.721911 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed98fdb-8242-4cdb-a134-d0666db7e768 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:16.722355 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed98fdb-8242-4cdb-a134-d0666db7e768 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:16.722449 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed98fdb-8242-4cdb-a134-d0666db7e768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:16.741736 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-fed98fdb-8242-4cdb-a134-d0666db7e768 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:20:16.742863 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-fed98fdb-8242-4cdb-a134-d0666db7e768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:20:16.802000 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1159/2302] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:20:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:20:16.810580 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e8f2eb-6008-4404-9b14-d627dabd6a6f None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:16.810869 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e8f2eb-6008-4404-9b14-d627dabd6a6f None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:16.811149 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e8f2eb-6008-4404-9b14-d627dabd6a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:16.831142 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-94e8f2eb-6008-4404-9b14-d627dabd6a6f None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:20:16.832352 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-94e8f2eb-6008-4404-9b14-d627dabd6a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:20:16.840177 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1144/2303] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:20:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:20:16.844284 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed98fdb-8242-4cdb-a134-d0666db7e768 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:16.854344 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed98fdb-8242-4cdb-a134-d0666db7e768 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:16.858816 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ca3170-b0be-4f72-826f-34fe90a923e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:16.859400 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ca3170-b0be-4f72-826f-34fe90a923e3 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:16.859680 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ca3170-b0be-4f72-826f-34fe90a923e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:16.859952 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ca3170-b0be-4f72-826f-34fe90a923e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:16.860581 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ca3170-b0be-4f72-826f-34fe90a923e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 09:20:16.860882 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ca3170-b0be-4f72-826f-34fe90a923e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:16.862496 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ca3170-b0be-4f72-826f-34fe90a923e3 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:16.871485 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:20:16.874837 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1160/2304] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 09:20:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 09:20:16.878666 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e8f2eb-6008-4404-9b14-d627dabd6a6f None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:16.887807 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e8f2eb-6008-4404-9b14-d627dabd6a6f None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:16.892445 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0a83d0-5016-4c64-b0fe-7d3e91d1ea98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:16.893033 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a83d0-5016-4c64-b0fe-7d3e91d1ea98 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:16.893314 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a83d0-5016-4c64-b0fe-7d3e91d1ea98 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:16.893591 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a83d0-5016-4c64-b0fe-7d3e91d1ea98 admin admin] PATH_INFO: `/v3/projects` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:16.894259 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0a83d0-5016-4c64-b0fe-7d3e91d1ea98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:16.896163 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0a83d0-5016-4c64-b0fe-7d3e91d1ea98 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:16.923166 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1145/2305] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 09:20:16 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:20:16.927249 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ca3170-b0be-4f72-826f-34fe90a923e3 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:16.937533 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ca3170-b0be-4f72-826f-34fe90a923e3 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:16.992597 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333e09f6-87e6-4bba-b3b7-7b009b45228f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:16.993206 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e09f6-87e6-4bba-b3b7-7b009b45228f admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:16.993489 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e09f6-87e6-4bba-b3b7-7b009b45228f admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:16.994365 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e09f6-87e6-4bba-b3b7-7b009b45228f admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:16.995000 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333e09f6-87e6-4bba-b3b7-7b009b45228f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:16.996911 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333e09f6-87e6-4bba-b3b7-7b009b45228f admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:17.018092 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1161/2306] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 09:20:16 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 09:20:17.025840 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0a83d0-5016-4c64-b0fe-7d3e91d1ea98 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:17.032540 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0a83d0-5016-4c64-b0fe-7d3e91d1ea98 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:17.037208 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe0bf33-0a78-4e1a-954c-b5de0ecff0dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:17.037835 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe0bf33-0a78-4e1a-954c-b5de0ecff0dd admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:17.038111 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe0bf33-0a78-4e1a-954c-b5de0ecff0dd admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:17.038380 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe0bf33-0a78-4e1a-954c-b5de0ecff0dd admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:17.039012 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe0bf33-0a78-4e1a-954c-b5de0ecff0dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:17.041112 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe0bf33-0a78-4e1a-954c-b5de0ecff0dd admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:17.051004 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1146/2307] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:20:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 09:20:17.054913 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333e09f6-87e6-4bba-b3b7-7b009b45228f admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:17.065557 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333e09f6-87e6-4bba-b3b7-7b009b45228f admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:17.070179 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d26a687-998b-45e3-a1e6-ebc1a031e8b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:17.070777 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d26a687-998b-45e3-a1e6-ebc1a031e8b8 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:17.071063 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d26a687-998b-45e3-a1e6-ebc1a031e8b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:17.071341 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d26a687-998b-45e3-a1e6-ebc1a031e8b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:17.072071 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d26a687-998b-45e3-a1e6-ebc1a031e8b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:17.073701 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d26a687-998b-45e3-a1e6-ebc1a031e8b8 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:17.084140 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1162/2308] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:20:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:20:17.088149 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe0bf33-0a78-4e1a-954c-b5de0ecff0dd admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:17.098175 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe0bf33-0a78-4e1a-954c-b5de0ecff0dd admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:17.102716 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f9310e-1f40-49bb-90db-e29344287820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:17.103926 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f9310e-1f40-49bb-90db-e29344287820 admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:17.104214 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f9310e-1f40-49bb-90db-e29344287820 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:17.104630 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f9310e-1f40-49bb-90db-e29344287820 admin admin] PATH_INFO: `/v3/projects/38ddbfdd17554df08168aa21eba90a89/users/b18160f075cd4c6f828c92010b30a98a/roles/3248f0268738489b883041851d681bb7` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:17.119431 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f9310e-1f40-49bb-90db-e29344287820 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38ddbfdd17554df08168aa21eba90a89, user_id=b18160f075cd4c6f828c92010b30a98a, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:17.122109 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f9310e-1f40-49bb-90db-e29344287820 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:17.134694 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1147/2309] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:20:17 2022] PUT /identity/v3/projects/38ddbfdd17554df08168aa21eba90a89/users/b18160f075cd4c6f828c92010b30a98a/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:20:17.146035 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8bfd6-1831-4977-86bb-0cb7348cb5ce None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:17.146326 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8bfd6-1831-4977-86bb-0cb7348cb5ce None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:17.146607 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8bfd6-1831-4977-86bb-0cb7348cb5ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:17.174199 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-2cf8bfd6-1831-4977-86bb-0cb7348cb5ce None None] MFA Rules not processed for user `b18160f075cd4c6f828c92010b30a98a`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:20:17.175408 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-2cf8bfd6-1831-4977-86bb-0cb7348cb5ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:20:17.238697 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1163/2310] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:20:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:20:17.249693 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f9310e-1f40-49bb-90db-e29344287820 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:17.251401 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-b6f9310e-1f40-49bb-90db-e29344287820 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:20:17.262363 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:INSERT=4,keystone:DELETE=10,keystone:SELECT=86 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:20:17.311742 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f9310e-1f40-49bb-90db-e29344287820 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:17.315630 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75a9f99-9e45-47a7-921e-aed262e05e64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:17.316232 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75a9f99-9e45-47a7-921e-aed262e05e64 service cinder] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:17.316515 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75a9f99-9e45-47a7-921e-aed262e05e64 service cinder] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:17.316790 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75a9f99-9e45-47a7-921e-aed262e05e64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:17.325061 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75a9f99-9e45-47a7-921e-aed262e05e64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:17.326519 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75a9f99-9e45-47a7-921e-aed262e05e64 service cinder] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:17.335050 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1148/2311] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:20:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:20:27.235880 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=66,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:20:27.331122 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=22 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:20:35.881797 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8bfd6-1831-4977-86bb-0cb7348cb5ce None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:35.945706 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8bfd6-1831-4977-86bb-0cb7348cb5ce None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:35.951076 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b3214e-fcd3-412e-9ad2-332a4a380cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:35.951714 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b3214e-fcd3-412e-9ad2-332a4a380cfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:35.951997 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b3214e-fcd3-412e-9ad2-332a4a380cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:35.952271 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b3214e-fcd3-412e-9ad2-332a4a380cfc admin admin] PATH_INFO: `/v3/users/b18160f075cd4c6f828c92010b30a98a` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:35.953414 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b3214e-fcd3-412e-9ad2-332a4a380cfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=b18160f075cd4c6f828c92010b30a98a)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:35.955368 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b3214e-fcd3-412e-9ad2-332a4a380cfc admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:36.002947 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-f1b3214e-fcd3-412e-9ad2-332a4a380cfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b18160f075cd4c6f828c92010b30a98a'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:36.003599 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-f1b3214e-fcd3-412e-9ad2-332a4a380cfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b18160f075cd4c6f828c92010b30a98a'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:36.010625 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-f1b3214e-fcd3-412e-9ad2-332a4a380cfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b18160f075cd4c6f828c92010b30a98a'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:36.022998 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-f1b3214e-fcd3-412e-9ad2-332a4a380cfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b18160f075cd4c6f828c92010b30a98a'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:36.030114 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-f1b3214e-fcd3-412e-9ad2-332a4a380cfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b18160f075cd4c6f828c92010b30a98a'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:36.036137 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1164/2312] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 09:20:35 2022] DELETE /identity/v3/users/b18160f075cd4c6f828c92010b30a98a => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:20:36.040580 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975cebe2-4c46-41a8-865f-7fc9865f8018 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:36.041100 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975cebe2-4c46-41a8-865f-7fc9865f8018 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:36.041100 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975cebe2-4c46-41a8-865f-7fc9865f8018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:36.060940 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-975cebe2-4c46-41a8-865f-7fc9865f8018 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:20:36.062169 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-975cebe2-4c46-41a8-865f-7fc9865f8018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:20:36.118650 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1149/2313] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:20:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:20:36.128674 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b3214e-fcd3-412e-9ad2-332a4a380cfc admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:36.130096 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-f1b3214e-fcd3-412e-9ad2-332a4a380cfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:20:36.187334 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b3214e-fcd3-412e-9ad2-332a4a380cfc admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:36.190928 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef51ca93-1751-40bf-8ee3-1ffacc1cecbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:36.191527 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef51ca93-1751-40bf-8ee3-1ffacc1cecbd service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:36.191839 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef51ca93-1751-40bf-8ee3-1ffacc1cecbd service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:36.192110 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef51ca93-1751-40bf-8ee3-1ffacc1cecbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:36.201505 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef51ca93-1751-40bf-8ee3-1ffacc1cecbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:36.202940 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef51ca93-1751-40bf-8ee3-1ffacc1cecbd service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:36.212513 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1165/2314] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:20:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:20:36.474165 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975cebe2-4c46-41a8-865f-7fc9865f8018 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:36.475612 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-975cebe2-4c46-41a8-865f-7fc9865f8018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:20:36.488265 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975cebe2-4c46-41a8-865f-7fc9865f8018 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:36.492973 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a21cd82-ff0b-4434-926e-0628ec6d272b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:36.493610 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a21cd82-ff0b-4434-926e-0628ec6d272b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:36.493929 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a21cd82-ff0b-4434-926e-0628ec6d272b admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:36.494233 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a21cd82-ff0b-4434-926e-0628ec6d272b admin admin] PATH_INFO: `/v3/projects/38ddbfdd17554df08168aa21eba90a89` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:36.495367 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a21cd82-ff0b-4434-926e-0628ec6d272b admin admin] RBAC: Authorizing `identity:delete_project(project_id=38ddbfdd17554df08168aa21eba90a89)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:36.497884 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a21cd82-ff0b-4434-926e-0628ec6d272b admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:36.550771 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-9a21cd82-ff0b-4434-926e-0628ec6d272b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38ddbfdd17554df08168aa21eba90a89', 'request_id': 'req-9a21cd82-ff0b-4434-926e-0628ec6d272b'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:36.557381 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-9a21cd82-ff0b-4434-926e-0628ec6d272b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38ddbfdd17554df08168aa21eba90a89', 'request_id': 'req-9a21cd82-ff0b-4434-926e-0628ec6d272b'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:36.565655 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-9a21cd82-ff0b-4434-926e-0628ec6d272b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38ddbfdd17554df08168aa21eba90a89', 'request_id': 'req-9a21cd82-ff0b-4434-926e-0628ec6d272b'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:36.576528 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-9a21cd82-ff0b-4434-926e-0628ec6d272b admin admin] The token cache is being invalidate because project 38ddbfdd17554df08168aa21eba90a89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 09:20:36.576805 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-9a21cd82-ff0b-4434-926e-0628ec6d272b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:36.577465 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-9a21cd82-ff0b-4434-926e-0628ec6d272b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:20:36.578979 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1150/2315] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 09:20:36 2022] DELETE /identity/v3/projects/38ddbfdd17554df08168aa21eba90a89 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:20:36.583777 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6aeaa4-6b4d-4423-a65c-4b37d30f822f None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:36.584070 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6aeaa4-6b4d-4423-a65c-4b37d30f822f None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:36.584352 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6aeaa4-6b4d-4423-a65c-4b37d30f822f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:36.603957 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-da6aeaa4-6b4d-4423-a65c-4b37d30f822f None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:20:36.605158 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-da6aeaa4-6b4d-4423-a65c-4b37d30f822f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:20:36.622060 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:20:36.664579 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1166/2316] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:20:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:20:36.672880 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7809985e-2a4c-4ae0-8dc5-655572e5ac65 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:36.673425 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7809985e-2a4c-4ae0-8dc5-655572e5ac65 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:36.673736 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7809985e-2a4c-4ae0-8dc5-655572e5ac65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:36.693073 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-7809985e-2a4c-4ae0-8dc5-655572e5ac65 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:20:36.694304 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-7809985e-2a4c-4ae0-8dc5-655572e5ac65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:20:36.702638 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1151/2317] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:20:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 09:20:36.706897 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6aeaa4-6b4d-4423-a65c-4b37d30f822f None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:36.717298 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6aeaa4-6b4d-4423-a65c-4b37d30f822f None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:36.722108 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445ab21a-ae7e-41c1-b466-5b49988bf8aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:36.722711 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ab21a-ae7e-41c1-b466-5b49988bf8aa admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:36.722988 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ab21a-ae7e-41c1-b466-5b49988bf8aa admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:36.723263 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ab21a-ae7e-41c1-b466-5b49988bf8aa admin admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:36.723902 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445ab21a-ae7e-41c1-b466-5b49988bf8aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 09:20:36.724201 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445ab21a-ae7e-41c1-b466-5b49988bf8aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:36.725871 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445ab21a-ae7e-41c1-b466-5b49988bf8aa admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:36.738302 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1167/2318] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 09:20:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 09:20:36.742574 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7809985e-2a4c-4ae0-8dc5-655572e5ac65 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:36.751983 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7809985e-2a4c-4ae0-8dc5-655572e5ac65 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:36.756993 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e12973-5b56-4b08-823e-b135b682f6c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:36.757813 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e12973-5b56-4b08-823e-b135b682f6c9 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:36.758267 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e12973-5b56-4b08-823e-b135b682f6c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:36.758682 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e12973-5b56-4b08-823e-b135b682f6c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:36.759423 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e12973-5b56-4b08-823e-b135b682f6c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:36.761828 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e12973-5b56-4b08-823e-b135b682f6c9 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:36.788178 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1152/2319] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 09:20:36 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:20:36.792281 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445ab21a-ae7e-41c1-b466-5b49988bf8aa admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:36.803125 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445ab21a-ae7e-41c1-b466-5b49988bf8aa admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:36.858388 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89328530-1fed-4d75-90af-c3942f476672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:36.858980 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89328530-1fed-4d75-90af-c3942f476672 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:36.859264 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89328530-1fed-4d75-90af-c3942f476672 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:36.859540 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89328530-1fed-4d75-90af-c3942f476672 admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:36.860156 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89328530-1fed-4d75-90af-c3942f476672 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:36.862105 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89328530-1fed-4d75-90af-c3942f476672 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:36.882442 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1168/2320] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 09:20:36 2022] POST /identity/v3/users => generated 286 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:20:36.886535 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e12973-5b56-4b08-823e-b135b682f6c9 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:36.896580 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e12973-5b56-4b08-823e-b135b682f6c9 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:36.901054 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a36deb5-0637-4a28-a2ef-cb23f94c133a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:36.901642 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a36deb5-0637-4a28-a2ef-cb23f94c133a admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:36.901950 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a36deb5-0637-4a28-a2ef-cb23f94c133a admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:36.902228 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a36deb5-0637-4a28-a2ef-cb23f94c133a admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:36.902860 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a36deb5-0637-4a28-a2ef-cb23f94c133a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:36.905013 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a36deb5-0637-4a28-a2ef-cb23f94c133a admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:36.914619 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1153/2321] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:20:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:20:36.918272 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89328530-1fed-4d75-90af-c3942f476672 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:36.929144 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89328530-1fed-4d75-90af-c3942f476672 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:36.934075 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e1236-f704-422d-b34a-38d6aedbc168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:36.934691 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e1236-f704-422d-b34a-38d6aedbc168 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:36.934974 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e1236-f704-422d-b34a-38d6aedbc168 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:36.935277 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e1236-f704-422d-b34a-38d6aedbc168 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:36.935912 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91e1236-f704-422d-b34a-38d6aedbc168 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:36.937494 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91e1236-f704-422d-b34a-38d6aedbc168 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:36.947878 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1169/2322] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:20:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:20:36.951761 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a36deb5-0637-4a28-a2ef-cb23f94c133a admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:36.961458 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a36deb5-0637-4a28-a2ef-cb23f94c133a admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:36.966484 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c4a84-a6d0-4cc7-b504-7f9c27f6081f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:36.967134 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c4a84-a6d0-4cc7-b504-7f9c27f6081f admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:36.967412 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c4a84-a6d0-4cc7-b504-7f9c27f6081f admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:36.967668 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c4a84-a6d0-4cc7-b504-7f9c27f6081f admin admin] PATH_INFO: `/v3/projects/0c592eb488aa4949a99f2eae3be6b2fa/users/529f960930634bc6b0cec7093f0f2e37/roles/3248f0268738489b883041851d681bb7` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:36.982242 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686c4a84-a6d0-4cc7-b504-7f9c27f6081f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c592eb488aa4949a99f2eae3be6b2fa, user_id=529f960930634bc6b0cec7093f0f2e37, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:36.984871 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686c4a84-a6d0-4cc7-b504-7f9c27f6081f admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:36.997661 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1154/2323] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:20:36 2022] PUT /identity/v3/projects/0c592eb488aa4949a99f2eae3be6b2fa/users/529f960930634bc6b0cec7093f0f2e37/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:20:37.002570 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29bd97b-57e4-4dae-b430-50d6a051d960 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:37.002862 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29bd97b-57e4-4dae-b430-50d6a051d960 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:37.003142 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29bd97b-57e4-4dae-b430-50d6a051d960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:37.022916 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-b29bd97b-57e4-4dae-b430-50d6a051d960 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:20:37.024113 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-b29bd97b-57e4-4dae-b430-50d6a051d960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:20:37.082341 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1170/2324] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:20:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:20:37.092263 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c4a84-a6d0-4cc7-b504-7f9c27f6081f admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:37.093989 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-686c4a84-a6d0-4cc7-b504-7f9c27f6081f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:20:37.115729 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:20:37.153314 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c4a84-a6d0-4cc7-b504-7f9c27f6081f admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:37.157103 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3d7585-accf-4e9a-ba06-adc4d122dd01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:37.157731 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d7585-accf-4e9a-ba06-adc4d122dd01 service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:37.158036 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d7585-accf-4e9a-ba06-adc4d122dd01 service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:37.158312 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d7585-accf-4e9a-ba06-adc4d122dd01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:37.167560 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3d7585-accf-4e9a-ba06-adc4d122dd01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:37.169562 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3d7585-accf-4e9a-ba06-adc4d122dd01 service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:37.179173 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1155/2325] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:20:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:20:37.799088 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1c1bd1-2602-4294-b7ae-57d607b929ef None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:37.799383 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1c1bd1-2602-4294-b7ae-57d607b929ef None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:37.799664 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1c1bd1-2602-4294-b7ae-57d607b929ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:37.827726 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-ca1c1bd1-2602-4294-b7ae-57d607b929ef None None] MFA Rules not processed for user `529f960930634bc6b0cec7093f0f2e37`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:20:37.828957 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-ca1c1bd1-2602-4294-b7ae-57d607b929ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:20:37.847190 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:20:37.897060 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1171/2326] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:20:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:20:37.908286 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3d7585-accf-4e9a-ba06-adc4d122dd01 service neutron] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:37.909818 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-2f3d7585-accf-4e9a-ba06-adc4d122dd01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:20:37.969748 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3d7585-accf-4e9a-ba06-adc4d122dd01 service neutron] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:37.973798 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90902289-856e-4e57-aa9d-b9da5e7a8f25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:37.974404 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90902289-856e-4e57-aa9d-b9da5e7a8f25 service cinder] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:37.974686 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90902289-856e-4e57-aa9d-b9da5e7a8f25 service cinder] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:37.974966 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90902289-856e-4e57-aa9d-b9da5e7a8f25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:37.983517 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90902289-856e-4e57-aa9d-b9da5e7a8f25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:37.985021 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90902289-856e-4e57-aa9d-b9da5e7a8f25 service cinder] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:37.993929 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1156/2327] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:20:37 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:20:39.346434 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f0107-18f9-4985-9084-f4603529b054 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:39.346934 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f0107-18f9-4985-9084-f4603529b054 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:39.347045 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f0107-18f9-4985-9084-f4603529b054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:39.365859 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-2a2f0107-18f9-4985-9084-f4603529b054 None None] MFA Rules not processed for user `529f960930634bc6b0cec7093f0f2e37`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:20:39.366984 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-2a2f0107-18f9-4985-9084-f4603529b054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:20:39.373721 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1172/2328] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:20:39 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:20:39.607320 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90902289-856e-4e57-aa9d-b9da5e7a8f25 service cinder] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:39.608766 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-90902289-856e-4e57-aa9d-b9da5e7a8f25 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:20:39.620520 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90902289-856e-4e57-aa9d-b9da5e7a8f25 service cinder] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:39.624496 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b45135-5014-419a-b307-a7ff2880d1d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:39.625084 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b45135-5014-419a-b307-a7ff2880d1d9 service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:39.625363 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b45135-5014-419a-b307-a7ff2880d1d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:39.625636 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b45135-5014-419a-b307-a7ff2880d1d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:39.627282 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-50b45135-5014-419a-b307-a7ff2880d1d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:20:39.636711 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b45135-5014-419a-b307-a7ff2880d1d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:39.638824 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b45135-5014-419a-b307-a7ff2880d1d9 service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:39.646640 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1157/2329] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:20:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:20:48.652359 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2f0107-18f9-4985-9084-f4603529b054 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:48.661991 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2f0107-18f9-4985-9084-f4603529b054 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:48.665463 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83732fd3-0de2-4ecb-91aa-3043d8c31a3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:48.666154 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83732fd3-0de2-4ecb-91aa-3043d8c31a3c service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:48.666439 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83732fd3-0de2-4ecb-91aa-3043d8c31a3c service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:48.666713 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83732fd3-0de2-4ecb-91aa-3043d8c31a3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:48.668452 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-83732fd3-0de2-4ecb-91aa-3043d8c31a3c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:20:48.740493 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83732fd3-0de2-4ecb-91aa-3043d8c31a3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:48.741956 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83732fd3-0de2-4ecb-91aa-3043d8c31a3c service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:48.751500 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1173/2330] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:20:48 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:20:49.133117 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b45135-5014-419a-b307-a7ff2880d1d9 service neutron] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:49.142414 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b45135-5014-419a-b307-a7ff2880d1d9 service neutron] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:49.146432 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f3e55-5600-4dd9-b7a1-c8331a7a2118 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:49.147098 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f3e55-5600-4dd9-b7a1-c8331a7a2118 service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:49.147382 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f3e55-5600-4dd9-b7a1-c8331a7a2118 service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:49.147656 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f3e55-5600-4dd9-b7a1-c8331a7a2118 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:49.149355 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-cf1f3e55-5600-4dd9-b7a1-c8331a7a2118 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:20:49.207837 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83732fd3-0de2-4ecb-91aa-3043d8c31a3c service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:20:49.209493 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-83732fd3-0de2-4ecb-91aa-3043d8c31a3c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:20:49.221556 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83732fd3-0de2-4ecb-91aa-3043d8c31a3c service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:20:49.224470 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1f3e55-5600-4dd9-b7a1-c8331a7a2118 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:49.225825 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d952b-f63b-42c6-83f1-878675261dfb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:20:49.225963 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1f3e55-5600-4dd9-b7a1-c8331a7a2118 service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:49.226701 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d952b-f63b-42c6-83f1-878675261dfb service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:49.226992 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d952b-f63b-42c6-83f1-878675261dfb service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:49.227279 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d952b-f63b-42c6-83f1-878675261dfb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:49.229055 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-e88d952b-f63b-42c6-83f1-878675261dfb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:20:49.237937 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1158/2331] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:20:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:20:49.314613 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88d952b-f63b-42c6-83f1-878675261dfb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:20:49.316058 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88d952b-f63b-42c6-83f1-878675261dfb service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:20:49.335266 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1174/2332] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:20:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:20:55.831244 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fd13af-c41c-4e9d-9562-f95986ea9913 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:20:55.831717 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fd13af-c41c-4e9d-9562-f95986ea9913 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:20:55.831815 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fd13af-c41c-4e9d-9562-f95986ea9913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:20:55.851045 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-79fd13af-c41c-4e9d-9562-f95986ea9913 None None] MFA Rules not processed for user `529f960930634bc6b0cec7093f0f2e37`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:20:55.852169 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-79fd13af-c41c-4e9d-9562-f95986ea9913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:20:55.858951 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1159/2333] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:20:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 09:20:59.332472 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=96 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:21:05.845497 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=86 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:21:12.366114 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a14f18-4a43-480f-ad32-b89016bf2530 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:21:12.366614 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a14f18-4a43-480f-ad32-b89016bf2530 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:21:12.366717 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a14f18-4a43-480f-ad32-b89016bf2530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:21:12.386524 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-f1a14f18-4a43-480f-ad32-b89016bf2530 None None] MFA Rules not processed for user `c5b2a0e623c348ad94d4820080677be6`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:21:12.387745 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-f1a14f18-4a43-480f-ad32-b89016bf2530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:21:12.443178 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1175/2334] 10.222.0.30 () {58 vars in 1009 bytes} [Sat Oct 15 09:21:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:21:12.487575 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fd13af-c41c-4e9d-9562-f95986ea9913 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:21:12.488968 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-79fd13af-c41c-4e9d-9562-f95986ea9913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:21:12.560987 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fd13af-c41c-4e9d-9562-f95986ea9913 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:21:12.566002 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03934438-b019-4e12-8484-9e01d1463347 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e3009e20e4c49e5a74f7fd793b2c329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:21:12.566617 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03934438-b019-4e12-8484-9e01d1463347 service swift] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:21:12.566902 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03934438-b019-4e12-8484-9e01d1463347 service swift] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:21:12.567185 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03934438-b019-4e12-8484-9e01d1463347 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:21:12.575369 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03934438-b019-4e12-8484-9e01d1463347 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:21:12.577164 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03934438-b019-4e12-8484-9e01d1463347 service swift] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:21:12.586166 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1160/2335] 10.222.0.30 () {60 vars in 1384 bytes} [Sat Oct 15 09:21:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 15 09:21:22.436630 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=25 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:21:22.583239 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=32 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:21:28.015375 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158a136-9a54-4d84-983c-5863769e0548 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:21:28.016029 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158a136-9a54-4d84-983c-5863769e0548 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:21:28.016029 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158a136-9a54-4d84-983c-5863769e0548 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:21:28.017310 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1176/2336] 10.222.0.30 () {54 vars in 921 bytes} [Sat Oct 15 09:21:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 09:21:28.023766 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ff9509-240d-4479-acb3-721fbbd470b3 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:21:28.024242 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ff9509-240d-4479-acb3-721fbbd470b3 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:21:28.024711 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ff9509-240d-4479-acb3-721fbbd470b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:21:28.069771 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-10ff9509-240d-4479-acb3-721fbbd470b3 None None] MFA Rules not processed for user `8b11e349e4cb4558a4c75d843effc0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:21:28.070988 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-10ff9509-240d-4479-acb3-721fbbd470b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:21:28.078351 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1161/2337] 10.222.0.30 () {58 vars in 1018 bytes} [Sat Oct 15 09:21:28 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 09:21:28.092710 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b158a136-9a54-4d84-983c-5863769e0548 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:21:28.095762 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea342a4-4bfd-4ea2-afa3-8638fd90ce98 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:21:28.096325 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea342a4-4bfd-4ea2-afa3-8638fd90ce98 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:21:28.096758 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea342a4-4bfd-4ea2-afa3-8638fd90ce98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:21:28.104283 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b158a136-9a54-4d84-983c-5863769e0548 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:21:28.110168 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0912b9e-a446-4e7f-b945-dbefa8d05254 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:21:28.110739 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0912b9e-a446-4e7f-b945-dbefa8d05254 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:21:28.111040 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0912b9e-a446-4e7f-b945-dbefa8d05254 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:21:28.111296 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0912b9e-a446-4e7f-b945-dbefa8d05254 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:21:28.118317 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-7ea342a4-4bfd-4ea2-afa3-8638fd90ce98 None None] MFA Rules not processed for user `c5b2a0e623c348ad94d4820080677be6`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:21:28.119658 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-7ea342a4-4bfd-4ea2-afa3-8638fd90ce98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:21:28.119923 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0912b9e-a446-4e7f-b945-dbefa8d05254 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:21:28.121360 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0912b9e-a446-4e7f-b945-dbefa8d05254 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:21:28.127770 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1162/2338] 10.222.0.30 () {58 vars in 967 bytes} [Sat Oct 15 09:21:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 09:21:28.130084 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1177/2339] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:21:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:21:28.141694 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea342a4-4bfd-4ea2-afa3-8638fd90ce98 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:21:28.154966 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea342a4-4bfd-4ea2-afa3-8638fd90ce98 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:21:28.160486 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3f5bf5-7eba-4078-9690-c7fc481a57fa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e3009e20e4c49e5a74f7fd793b2c329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:21:28.161202 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f5bf5-7eba-4078-9690-c7fc481a57fa service swift] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:21:28.161552 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f5bf5-7eba-4078-9690-c7fc481a57fa service swift] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:21:28.161921 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f5bf5-7eba-4078-9690-c7fc481a57fa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:21:28.170295 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3f5bf5-7eba-4078-9690-c7fc481a57fa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:21:28.171969 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3f5bf5-7eba-4078-9690-c7fc481a57fa service swift] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:21:28.180611 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1163/2340] 10.222.0.30 () {60 vars in 1384 bytes} [Sat Oct 15 09:21:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 09:21:28.220820 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb3c4ee-0c71-4d8b-87ef-feecdf0e8c50 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:21:28.221172 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb3c4ee-0c71-4d8b-87ef-feecdf0e8c50 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:21:28.221517 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb3c4ee-0c71-4d8b-87ef-feecdf0e8c50 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:21:28.223059 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1178/2341] 10.222.0.30 () {54 vars in 921 bytes} [Sat Oct 15 09:21:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 09:21:28.228565 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104186f1-465e-4478-91ad-92554ab57d3a None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:21:28.228852 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104186f1-465e-4478-91ad-92554ab57d3a None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:21:28.229125 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104186f1-465e-4478-91ad-92554ab57d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:21:28.269756 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-104186f1-465e-4478-91ad-92554ab57d3a None None] MFA Rules not processed for user `af0a8af5cadc4cc1ab73e8eb2a20bb7e`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:21:28.271139 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-104186f1-465e-4478-91ad-92554ab57d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:21:28.281350 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1164/2342] 10.222.0.30 () {58 vars in 1018 bytes} [Sat Oct 15 09:21:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:21:28.292385 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb3c4ee-0c71-4d8b-87ef-feecdf0e8c50 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:21:28.305147 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb3c4ee-0c71-4d8b-87ef-feecdf0e8c50 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:21:28.308760 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b46b93-5f65-4a1d-aabc-4fe67e74aa6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:21:28.309361 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b46b93-5f65-4a1d-aabc-4fe67e74aa6f service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:21:28.309648 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b46b93-5f65-4a1d-aabc-4fe67e74aa6f service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:21:28.310532 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b46b93-5f65-4a1d-aabc-4fe67e74aa6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:21:28.321769 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b46b93-5f65-4a1d-aabc-4fe67e74aa6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:21:28.323274 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b46b93-5f65-4a1d-aabc-4fe67e74aa6f service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:21:28.336165 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1179/2343] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:21:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:21:30.144019 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104186f1-465e-4478-91ad-92554ab57d3a None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:21:30.155148 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104186f1-465e-4478-91ad-92554ab57d3a None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:21:30.160131 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f99348f-79a1-4f40-a5a4-8a7ac9d6bcc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:21:30.160877 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f99348f-79a1-4f40-a5a4-8a7ac9d6bcc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:21:30.161263 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f99348f-79a1-4f40-a5a4-8a7ac9d6bcc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:21:30.161664 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f99348f-79a1-4f40-a5a4-8a7ac9d6bcc3 admin admin] PATH_INFO: `/v3/users/529f960930634bc6b0cec7093f0f2e37` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:21:30.162939 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f99348f-79a1-4f40-a5a4-8a7ac9d6bcc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=529f960930634bc6b0cec7093f0f2e37)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:21:30.165027 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f99348f-79a1-4f40-a5a4-8a7ac9d6bcc3 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:21:30.217042 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-8f99348f-79a1-4f40-a5a4-8a7ac9d6bcc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '529f960930634bc6b0cec7093f0f2e37'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:21:30.217726 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-8f99348f-79a1-4f40-a5a4-8a7ac9d6bcc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '529f960930634bc6b0cec7093f0f2e37'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:21:30.229846 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-8f99348f-79a1-4f40-a5a4-8a7ac9d6bcc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '529f960930634bc6b0cec7093f0f2e37'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:21:30.237629 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-8f99348f-79a1-4f40-a5a4-8a7ac9d6bcc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '529f960930634bc6b0cec7093f0f2e37'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:21:30.238316 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-8f99348f-79a1-4f40-a5a4-8a7ac9d6bcc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '529f960930634bc6b0cec7093f0f2e37'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:21:30.250631 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1165/2344] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 09:21:30 2022] DELETE /identity/v3/users/529f960930634bc6b0cec7093f0f2e37 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:21:30.427256 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b46b93-5f65-4a1d-aabc-4fe67e74aa6f service neutron] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:21:30.439982 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b46b93-5f65-4a1d-aabc-4fe67e74aa6f service neutron] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:21:30.499861 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d928ad-0f60-4246-81b7-58269954e4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:21:30.500491 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d928ad-0f60-4246-81b7-58269954e4d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:21:30.500770 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d928ad-0f60-4246-81b7-58269954e4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:21:30.501046 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d928ad-0f60-4246-81b7-58269954e4d3 admin admin] PATH_INFO: `/v3/projects/0c592eb488aa4949a99f2eae3be6b2fa` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:21:30.502193 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d928ad-0f60-4246-81b7-58269954e4d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c592eb488aa4949a99f2eae3be6b2fa)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:21:30.504076 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d928ad-0f60-4246-81b7-58269954e4d3 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:21:30.557445 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-68d928ad-0f60-4246-81b7-58269954e4d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c592eb488aa4949a99f2eae3be6b2fa', 'request_id': 'req-68d928ad-0f60-4246-81b7-58269954e4d3'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:21:30.564207 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-68d928ad-0f60-4246-81b7-58269954e4d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c592eb488aa4949a99f2eae3be6b2fa', 'request_id': 'req-68d928ad-0f60-4246-81b7-58269954e4d3'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:21:30.571627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-68d928ad-0f60-4246-81b7-58269954e4d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c592eb488aa4949a99f2eae3be6b2fa', 'request_id': 'req-68d928ad-0f60-4246-81b7-58269954e4d3'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:21:30.582731 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-68d928ad-0f60-4246-81b7-58269954e4d3 admin admin] The token cache is being invalidate because project 0c592eb488aa4949a99f2eae3be6b2fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 09:21:30.583021 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-68d928ad-0f60-4246-81b7-58269954e4d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:21:30.583664 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-68d928ad-0f60-4246-81b7-58269954e4d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:21:30.585172 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1180/2345] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 09:21:30 2022] DELETE /identity/v3/projects/0c592eb488aa4949a99f2eae3be6b2fa => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:21:40.247085 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=56,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:21:40.579639 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=63,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:21:50.280016 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f99348f-79a1-4f40-a5a4-8a7ac9d6bcc3 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:21:50.281493 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-8f99348f-79a1-4f40-a5a4-8a7ac9d6bcc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:21:50.338058 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f99348f-79a1-4f40-a5a4-8a7ac9d6bcc3 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:21:50.384930 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca5d958-8315-4d11-b5d1-fa66f96ce3b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:21:50.385536 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca5d958-8315-4d11-b5d1-fa66f96ce3b0 service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:21:50.385851 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca5d958-8315-4d11-b5d1-fa66f96ce3b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:21:50.386129 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca5d958-8315-4d11-b5d1-fa66f96ce3b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:21:50.387893 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-fca5d958-8315-4d11-b5d1-fa66f96ce3b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:21:50.449640 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca5d958-8315-4d11-b5d1-fa66f96ce3b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:21:50.451523 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca5d958-8315-4d11-b5d1-fa66f96ce3b0 service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:21:50.461063 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1166/2346] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:21:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:21:53.509147 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d928ad-0f60-4246-81b7-58269954e4d3 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:21:53.510804 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-68d928ad-0f60-4246-81b7-58269954e4d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:21:53.523833 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d928ad-0f60-4246-81b7-58269954e4d3 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:21:53.529465 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83766fa6-d1f4-46dd-a012-641e36cccfba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:21:53.530170 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83766fa6-d1f4-46dd-a012-641e36cccfba admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:21:53.530451 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83766fa6-d1f4-46dd-a012-641e36cccfba admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:21:53.530723 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83766fa6-d1f4-46dd-a012-641e36cccfba admin admin] PATH_INFO: `/v3/users/060d1fae9b0b488da1f7553cabd97d28` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:21:53.531847 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83766fa6-d1f4-46dd-a012-641e36cccfba admin admin] RBAC: Authorizing `identity:delete_user(user_id=060d1fae9b0b488da1f7553cabd97d28)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:21:53.533797 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83766fa6-d1f4-46dd-a012-641e36cccfba admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:21:53.580996 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-83766fa6-d1f4-46dd-a012-641e36cccfba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '060d1fae9b0b488da1f7553cabd97d28'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:21:53.581655 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-83766fa6-d1f4-46dd-a012-641e36cccfba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '060d1fae9b0b488da1f7553cabd97d28'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:21:53.588648 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-83766fa6-d1f4-46dd-a012-641e36cccfba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '060d1fae9b0b488da1f7553cabd97d28'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:21:53.600484 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-83766fa6-d1f4-46dd-a012-641e36cccfba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '060d1fae9b0b488da1f7553cabd97d28'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:21:53.608349 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-83766fa6-d1f4-46dd-a012-641e36cccfba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '060d1fae9b0b488da1f7553cabd97d28'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:21:53.614732 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1181/2347] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 09:21:53 2022] DELETE /identity/v3/users/060d1fae9b0b488da1f7553cabd97d28 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:21:53.783186 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca5d958-8315-4d11-b5d1-fa66f96ce3b0 service neutron] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:21:53.796052 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca5d958-8315-4d11-b5d1-fa66f96ce3b0 service neutron] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:21:53.854464 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb33ce9-bdd2-4042-8d55-d6e00d477426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:21:53.855090 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb33ce9-bdd2-4042-8d55-d6e00d477426 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:21:53.855372 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb33ce9-bdd2-4042-8d55-d6e00d477426 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:21:53.855647 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb33ce9-bdd2-4042-8d55-d6e00d477426 admin admin] PATH_INFO: `/v3/projects/ca38c1109acc49849daeb555b673e756` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:21:53.857308 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb33ce9-bdd2-4042-8d55-d6e00d477426 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca38c1109acc49849daeb555b673e756)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:21:53.859248 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb33ce9-bdd2-4042-8d55-d6e00d477426 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:21:53.866880 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=100 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:21:53.904701 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-ddb33ce9-bdd2-4042-8d55-d6e00d477426 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca38c1109acc49849daeb555b673e756', 'request_id': 'req-ddb33ce9-bdd2-4042-8d55-d6e00d477426'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:21:53.910913 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-ddb33ce9-bdd2-4042-8d55-d6e00d477426 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca38c1109acc49849daeb555b673e756', 'request_id': 'req-ddb33ce9-bdd2-4042-8d55-d6e00d477426'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:21:53.918448 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-ddb33ce9-bdd2-4042-8d55-d6e00d477426 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca38c1109acc49849daeb555b673e756', 'request_id': 'req-ddb33ce9-bdd2-4042-8d55-d6e00d477426'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:21:53.929447 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-ddb33ce9-bdd2-4042-8d55-d6e00d477426 admin admin] The token cache is being invalidate because project ca38c1109acc49849daeb555b673e756 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 09:21:53.929763 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-ddb33ce9-bdd2-4042-8d55-d6e00d477426 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:21:53.930383 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-ddb33ce9-bdd2-4042-8d55-d6e00d477426 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:21:53.931830 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1167/2348] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 09:21:53 2022] DELETE /identity/v3/projects/ca38c1109acc49849daeb555b673e756 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:21:53.937742 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c0ba2c-37b5-4eca-95a8-1e388a2a0078 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:21:53.938106 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c0ba2c-37b5-4eca-95a8-1e388a2a0078 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:21:53.938389 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c0ba2c-37b5-4eca-95a8-1e388a2a0078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:21:53.958151 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-11c0ba2c-37b5-4eca-95a8-1e388a2a0078 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:21:53.959369 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-11c0ba2c-37b5-4eca-95a8-1e388a2a0078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:21:54.027967 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1182/2349] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:21:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:21:54.036354 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39db49-9b84-425d-aa9c-0d698d3d820c None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:21:54.036648 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39db49-9b84-425d-aa9c-0d698d3d820c None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:21:54.036921 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39db49-9b84-425d-aa9c-0d698d3d820c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:21:54.058236 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-ae39db49-9b84-425d-aa9c-0d698d3d820c None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:21:54.059473 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-ae39db49-9b84-425d-aa9c-0d698d3d820c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:21:54.067555 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1168/2350] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:21:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:21:54.072674 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c0ba2c-37b5-4eca-95a8-1e388a2a0078 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:21:54.082522 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c0ba2c-37b5-4eca-95a8-1e388a2a0078 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:21:54.087272 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40363fe1-71dd-4c2e-ab78-816df306bf39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:21:54.087960 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40363fe1-71dd-4c2e-ab78-816df306bf39 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:21:54.088417 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40363fe1-71dd-4c2e-ab78-816df306bf39 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:21:54.088712 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40363fe1-71dd-4c2e-ab78-816df306bf39 admin admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:21:54.089497 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40363fe1-71dd-4c2e-ab78-816df306bf39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 09:21:54.091065 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40363fe1-71dd-4c2e-ab78-816df306bf39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:21:54.093288 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40363fe1-71dd-4c2e-ab78-816df306bf39 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:21:54.107791 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1183/2351] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 09:21:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 09:21:54.112211 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae39db49-9b84-425d-aa9c-0d698d3d820c None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:21:54.122958 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae39db49-9b84-425d-aa9c-0d698d3d820c None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:21:54.127777 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5152e316-a9ef-4c62-a4d7-025bb82879f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:21:54.128444 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5152e316-a9ef-4c62-a4d7-025bb82879f1 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:21:54.128792 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5152e316-a9ef-4c62-a4d7-025bb82879f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:21:54.129136 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5152e316-a9ef-4c62-a4d7-025bb82879f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:21:54.129875 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5152e316-a9ef-4c62-a4d7-025bb82879f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:21:54.131776 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5152e316-a9ef-4c62-a4d7-025bb82879f1 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:21:54.168476 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1169/2352] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 09:21:54 2022] POST /identity/v3/projects => generated 369 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:21:54.173293 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40363fe1-71dd-4c2e-ab78-816df306bf39 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:21:54.183227 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40363fe1-71dd-4c2e-ab78-816df306bf39 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:21:54.238917 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ef3cd7-b1b0-4e52-8aa4-f7700ebcdb55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:21:54.239446 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef3cd7-b1b0-4e52-8aa4-f7700ebcdb55 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:21:54.239732 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef3cd7-b1b0-4e52-8aa4-f7700ebcdb55 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:21:54.240009 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef3cd7-b1b0-4e52-8aa4-f7700ebcdb55 admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:21:54.240646 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ef3cd7-b1b0-4e52-8aa4-f7700ebcdb55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:21:54.242591 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ef3cd7-b1b0-4e52-8aa4-f7700ebcdb55 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:21:54.250439 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:21:54.265253 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1184/2353] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 09:21:54 2022] POST /identity/v3/users => generated 285 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 09:21:54.269274 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5152e316-a9ef-4c62-a4d7-025bb82879f1 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:21:54.279940 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5152e316-a9ef-4c62-a4d7-025bb82879f1 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:21:54.284441 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0cd851-1bd0-4b21-9d19-b11544f5d168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:21:54.285030 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0cd851-1bd0-4b21-9d19-b11544f5d168 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:21:54.285308 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0cd851-1bd0-4b21-9d19-b11544f5d168 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:21:54.285580 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0cd851-1bd0-4b21-9d19-b11544f5d168 admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:21:54.286264 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0cd851-1bd0-4b21-9d19-b11544f5d168 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:21:54.287836 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0cd851-1bd0-4b21-9d19-b11544f5d168 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:21:54.297494 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1170/2354] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:21:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:21:54.301351 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ef3cd7-b1b0-4e52-8aa4-f7700ebcdb55 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:21:54.311302 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ef3cd7-b1b0-4e52-8aa4-f7700ebcdb55 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:21:54.316871 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39427dc3-a1b5-4e17-8082-f191e28226a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:21:54.317475 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39427dc3-a1b5-4e17-8082-f191e28226a6 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:21:54.317794 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39427dc3-a1b5-4e17-8082-f191e28226a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:21:54.318071 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39427dc3-a1b5-4e17-8082-f191e28226a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:21:54.318742 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39427dc3-a1b5-4e17-8082-f191e28226a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:21:54.320364 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39427dc3-a1b5-4e17-8082-f191e28226a6 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:21:54.331551 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1185/2355] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:21:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:21:54.335528 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0cd851-1bd0-4b21-9d19-b11544f5d168 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:21:54.346009 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0cd851-1bd0-4b21-9d19-b11544f5d168 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:21:54.350676 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f755f9b1-8ebf-4405-8126-8ef367a40669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:21:54.351349 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f755f9b1-8ebf-4405-8126-8ef367a40669 admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:21:54.351633 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f755f9b1-8ebf-4405-8126-8ef367a40669 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:21:54.351911 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f755f9b1-8ebf-4405-8126-8ef367a40669 admin admin] PATH_INFO: `/v3/projects/fc3a1828a35a44598c25a7ffece3aa94/users/2703360ce30c44148e09ac9bb788011a/roles/3248f0268738489b883041851d681bb7` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:21:54.366506 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f755f9b1-8ebf-4405-8126-8ef367a40669 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc3a1828a35a44598c25a7ffece3aa94, user_id=2703360ce30c44148e09ac9bb788011a, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:21:54.369164 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f755f9b1-8ebf-4405-8126-8ef367a40669 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:21:54.381979 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1171/2356] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:21:54 2022] PUT /identity/v3/projects/fc3a1828a35a44598c25a7ffece3aa94/users/2703360ce30c44148e09ac9bb788011a/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:21:54.393663 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd33f5-2791-4f40-b895-a83db91a3a93 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:21:54.393975 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd33f5-2791-4f40-b895-a83db91a3a93 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:21:54.394262 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd33f5-2791-4f40-b895-a83db91a3a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:21:54.423163 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-b8dd33f5-2791-4f40-b895-a83db91a3a93 None None] MFA Rules not processed for user `2703360ce30c44148e09ac9bb788011a`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:21:54.424627 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-b8dd33f5-2791-4f40-b895-a83db91a3a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:21:54.486576 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1186/2357] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:21:54 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:21:54.491200 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f755f9b1-8ebf-4405-8126-8ef367a40669 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:21:54.555979 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f755f9b1-8ebf-4405-8126-8ef367a40669 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:21:54.561224 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081e8bb9-4649-4e2c-93fa-5f2f40cb712f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:21:54.561867 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081e8bb9-4649-4e2c-93fa-5f2f40cb712f admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:21:54.562151 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081e8bb9-4649-4e2c-93fa-5f2f40cb712f admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:21:54.562425 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081e8bb9-4649-4e2c-93fa-5f2f40cb712f admin admin] PATH_INFO: `/v3/projects` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:21:54.563065 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081e8bb9-4649-4e2c-93fa-5f2f40cb712f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:21:54.564977 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081e8bb9-4649-4e2c-93fa-5f2f40cb712f admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:21:54.591692 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1172/2358] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 09:21:54 2022] POST /identity/v3/projects => generated 367 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:21:54.595808 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dd33f5-2791-4f40-b895-a83db91a3a93 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:21:54.606528 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dd33f5-2791-4f40-b895-a83db91a3a93 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:21:54.662250 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7956eb-213f-441c-9e19-50ce9f3ec5c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:21:54.662848 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7956eb-213f-441c-9e19-50ce9f3ec5c1 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:21:54.663132 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7956eb-213f-441c-9e19-50ce9f3ec5c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:21:54.663407 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7956eb-213f-441c-9e19-50ce9f3ec5c1 admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:21:54.664046 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7956eb-213f-441c-9e19-50ce9f3ec5c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:21:54.665973 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7956eb-213f-441c-9e19-50ce9f3ec5c1 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:21:54.687867 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1187/2359] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 09:21:54 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:21:54.691871 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081e8bb9-4649-4e2c-93fa-5f2f40cb712f admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:21:54.702464 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081e8bb9-4649-4e2c-93fa-5f2f40cb712f admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:21:54.707153 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3961326c-6290-4384-81a2-598c4022fa84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:21:54.707770 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961326c-6290-4384-81a2-598c4022fa84 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:21:54.708050 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961326c-6290-4384-81a2-598c4022fa84 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:21:54.708326 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961326c-6290-4384-81a2-598c4022fa84 admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:21:54.709009 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3961326c-6290-4384-81a2-598c4022fa84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:21:54.710834 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3961326c-6290-4384-81a2-598c4022fa84 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:21:54.712900 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:21:54.722878 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1173/2360] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:21:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:21:54.726501 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7956eb-213f-441c-9e19-50ce9f3ec5c1 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:21:54.736422 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7956eb-213f-441c-9e19-50ce9f3ec5c1 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:21:54.741416 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ee5513-8039-4bf8-ae47-bbeb19a83c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:21:54.742179 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee5513-8039-4bf8-ae47-bbeb19a83c4a admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:21:54.742469 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee5513-8039-4bf8-ae47-bbeb19a83c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:21:54.742805 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee5513-8039-4bf8-ae47-bbeb19a83c4a admin admin] PATH_INFO: `/v3/projects/a167bab70f1d4050abc4d9069fb88e3d/users/371ecba49f954977bb792e43b1c8d107/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:21:54.758054 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ee5513-8039-4bf8-ae47-bbeb19a83c4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a167bab70f1d4050abc4d9069fb88e3d, user_id=371ecba49f954977bb792e43b1c8d107, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:21:54.760700 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ee5513-8039-4bf8-ae47-bbeb19a83c4a admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:21:54.774222 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1188/2361] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:21:54 2022] PUT /identity/v3/projects/a167bab70f1d4050abc4d9069fb88e3d/users/371ecba49f954977bb792e43b1c8d107/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:21:54.778052 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3961326c-6290-4384-81a2-598c4022fa84 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:21:54.787854 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3961326c-6290-4384-81a2-598c4022fa84 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:21:54.843473 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e20758-84cb-4fa3-9b7d-6bc0e8a723eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:21:54.844073 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e20758-84cb-4fa3-9b7d-6bc0e8a723eb admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:21:54.844354 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e20758-84cb-4fa3-9b7d-6bc0e8a723eb admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:21:54.844635 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e20758-84cb-4fa3-9b7d-6bc0e8a723eb admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:21:54.845266 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e20758-84cb-4fa3-9b7d-6bc0e8a723eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:21:54.846893 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e20758-84cb-4fa3-9b7d-6bc0e8a723eb admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:21:54.856436 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1174/2362] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:21:54 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:21:54.860486 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ee5513-8039-4bf8-ae47-bbeb19a83c4a admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:21:54.870610 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ee5513-8039-4bf8-ae47-bbeb19a83c4a admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:21:54.876021 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb06c9d3-3218-42e0-988f-fce22a156518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:21:54.876677 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb06c9d3-3218-42e0-988f-fce22a156518 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:21:54.876966 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb06c9d3-3218-42e0-988f-fce22a156518 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:21:54.877248 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb06c9d3-3218-42e0-988f-fce22a156518 admin admin] PATH_INFO: `/v3/projects/a167bab70f1d4050abc4d9069fb88e3d/users/371ecba49f954977bb792e43b1c8d107/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:21:54.879204 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb06c9d3-3218-42e0-988f-fce22a156518 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a167bab70f1d4050abc4d9069fb88e3d, user_id=371ecba49f954977bb792e43b1c8d107, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:21:54.881896 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb06c9d3-3218-42e0-988f-fce22a156518 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:21:54.892577 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1189/2363] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:21:54 2022] PUT /identity/v3/projects/a167bab70f1d4050abc4d9069fb88e3d/users/371ecba49f954977bb792e43b1c8d107/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:21:54.902072 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84045b48-03d9-4c23-8ae0-3691570fadc0 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:21:54.902358 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84045b48-03d9-4c23-8ae0-3691570fadc0 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:21:54.902634 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84045b48-03d9-4c23-8ae0-3691570fadc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:21:54.930388 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-84045b48-03d9-4c23-8ae0-3691570fadc0 None None] MFA Rules not processed for user `371ecba49f954977bb792e43b1c8d107`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:21:54.931585 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-84045b48-03d9-4c23-8ae0-3691570fadc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:21:54.999198 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1175/2364] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:21:54 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:21:55.009702 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb06c9d3-3218-42e0-988f-fce22a156518 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:21:55.011382 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-eb06c9d3-3218-42e0-988f-fce22a156518 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:21:55.033054 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:21:55.069633 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb06c9d3-3218-42e0-988f-fce22a156518 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:21:55.073397 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94e71c5-26ae-4055-94cd-0868bbe25293 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:21:55.074036 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e71c5-26ae-4055-94cd-0868bbe25293 service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:21:55.074316 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e71c5-26ae-4055-94cd-0868bbe25293 service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:21:55.074588 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e71c5-26ae-4055-94cd-0868bbe25293 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:21:55.083476 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94e71c5-26ae-4055-94cd-0868bbe25293 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:21:55.084889 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94e71c5-26ae-4055-94cd-0868bbe25293 service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:21:55.094087 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1190/2365] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:21:55 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 09:21:56.404601 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84045b48-03d9-4c23-8ae0-3691570fadc0 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:21:56.466846 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84045b48-03d9-4c23-8ae0-3691570fadc0 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:21:56.471345 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd58078-aa35-45f6-9217-994cf7e2c6ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:21:56.471967 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd58078-aa35-45f6-9217-994cf7e2c6ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:21:56.472254 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd58078-aa35-45f6-9217-994cf7e2c6ae admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:21:56.472533 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd58078-aa35-45f6-9217-994cf7e2c6ae admin admin] PATH_INFO: `/v3/users/2703360ce30c44148e09ac9bb788011a` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:21:56.473644 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd58078-aa35-45f6-9217-994cf7e2c6ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=2703360ce30c44148e09ac9bb788011a)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:21:56.475608 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd58078-aa35-45f6-9217-994cf7e2c6ae admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:21:56.496684 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:21:56.522763 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-bcd58078-aa35-45f6-9217-994cf7e2c6ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2703360ce30c44148e09ac9bb788011a'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:21:56.523358 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-bcd58078-aa35-45f6-9217-994cf7e2c6ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2703360ce30c44148e09ac9bb788011a'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:21:56.535629 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-bcd58078-aa35-45f6-9217-994cf7e2c6ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2703360ce30c44148e09ac9bb788011a'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:21:56.542382 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-bcd58078-aa35-45f6-9217-994cf7e2c6ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2703360ce30c44148e09ac9bb788011a'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:21:56.543019 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-bcd58078-aa35-45f6-9217-994cf7e2c6ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2703360ce30c44148e09ac9bb788011a'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:21:56.555429 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1176/2366] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 09:21:56 2022] DELETE /identity/v3/users/2703360ce30c44148e09ac9bb788011a => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:21:56.559832 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4158df-8991-4fe3-adbe-f1fb3ceb82db None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:21:56.560293 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4158df-8991-4fe3-adbe-f1fb3ceb82db None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:21:56.560388 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4158df-8991-4fe3-adbe-f1fb3ceb82db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:21:56.579845 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-3e4158df-8991-4fe3-adbe-f1fb3ceb82db None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:21:56.581082 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-3e4158df-8991-4fe3-adbe-f1fb3ceb82db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:21:56.638757 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1191/2367] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:21:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:21:56.648360 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd58078-aa35-45f6-9217-994cf7e2c6ae admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:21:56.649826 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-bcd58078-aa35-45f6-9217-994cf7e2c6ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:21:56.707777 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd58078-aa35-45f6-9217-994cf7e2c6ae admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:21:56.711568 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50063d6a-8180-4bf1-be94-4f11622b54a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:21:56.712228 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50063d6a-8180-4bf1-be94-4f11622b54a9 service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:21:56.712513 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50063d6a-8180-4bf1-be94-4f11622b54a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:21:56.712943 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50063d6a-8180-4bf1-be94-4f11622b54a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:21:56.723097 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50063d6a-8180-4bf1-be94-4f11622b54a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:21:56.724523 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50063d6a-8180-4bf1-be94-4f11622b54a9 service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:21:56.733972 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1177/2368] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:21:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:21:56.990411 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4158df-8991-4fe3-adbe-f1fb3ceb82db None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:21:56.991943 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-3e4158df-8991-4fe3-adbe-f1fb3ceb82db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:21:57.004841 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4158df-8991-4fe3-adbe-f1fb3ceb82db None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:21:57.010066 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0d1e88-4697-4dcd-9f66-5a16657a508f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:21:57.010728 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d1e88-4697-4dcd-9f66-5a16657a508f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:21:57.011063 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d1e88-4697-4dcd-9f66-5a16657a508f admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:21:57.011376 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d1e88-4697-4dcd-9f66-5a16657a508f admin admin] PATH_INFO: `/v3/projects/fc3a1828a35a44598c25a7ffece3aa94` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:21:57.013137 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0d1e88-4697-4dcd-9f66-5a16657a508f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc3a1828a35a44598c25a7ffece3aa94)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:21:57.015190 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0d1e88-4697-4dcd-9f66-5a16657a508f admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:21:57.061300 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8c0d1e88-4697-4dcd-9f66-5a16657a508f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc3a1828a35a44598c25a7ffece3aa94', 'request_id': 'req-8c0d1e88-4697-4dcd-9f66-5a16657a508f'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:21:57.067609 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8c0d1e88-4697-4dcd-9f66-5a16657a508f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc3a1828a35a44598c25a7ffece3aa94', 'request_id': 'req-8c0d1e88-4697-4dcd-9f66-5a16657a508f'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:21:57.075741 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8c0d1e88-4697-4dcd-9f66-5a16657a508f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc3a1828a35a44598c25a7ffece3aa94', 'request_id': 'req-8c0d1e88-4697-4dcd-9f66-5a16657a508f'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:21:57.086069 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8c0d1e88-4697-4dcd-9f66-5a16657a508f admin admin] The token cache is being invalidate because project fc3a1828a35a44598c25a7ffece3aa94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 09:21:57.086356 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8c0d1e88-4697-4dcd-9f66-5a16657a508f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:21:57.086978 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8c0d1e88-4697-4dcd-9f66-5a16657a508f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:21:57.088466 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1192/2369] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 09:21:56 2022] DELETE /identity/v3/projects/fc3a1828a35a44598c25a7ffece3aa94 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:21:57.092471 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50063d6a-8180-4bf1-be94-4f11622b54a9 service neutron] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:21:57.101869 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50063d6a-8180-4bf1-be94-4f11622b54a9 service neutron] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:21:57.156476 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad3464e-1535-4e9f-ab89-c2c558ce2707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:21:57.157083 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad3464e-1535-4e9f-ab89-c2c558ce2707 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:21:57.157360 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad3464e-1535-4e9f-ab89-c2c558ce2707 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:21:57.157628 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad3464e-1535-4e9f-ab89-c2c558ce2707 admin admin] PATH_INFO: `/v3/users/371ecba49f954977bb792e43b1c8d107` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:21:57.158746 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad3464e-1535-4e9f-ab89-c2c558ce2707 admin admin] RBAC: Authorizing `identity:delete_user(user_id=371ecba49f954977bb792e43b1c8d107)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:21:57.160645 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad3464e-1535-4e9f-ab89-c2c558ce2707 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:21:57.203266 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:21:57.206393 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-cad3464e-1535-4e9f-ab89-c2c558ce2707 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '371ecba49f954977bb792e43b1c8d107'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:21:57.207252 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-cad3464e-1535-4e9f-ab89-c2c558ce2707 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '371ecba49f954977bb792e43b1c8d107'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:21:57.220296 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-cad3464e-1535-4e9f-ab89-c2c558ce2707 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '371ecba49f954977bb792e43b1c8d107'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:21:57.227334 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-cad3464e-1535-4e9f-ab89-c2c558ce2707 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '371ecba49f954977bb792e43b1c8d107'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:21:57.227981 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-cad3464e-1535-4e9f-ab89-c2c558ce2707 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '371ecba49f954977bb792e43b1c8d107'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:21:57.240618 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1178/2370] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 09:21:57 2022] DELETE /identity/v3/users/371ecba49f954977bb792e43b1c8d107 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:21:57.509180 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0d1e88-4697-4dcd-9f66-5a16657a508f admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:21:57.510666 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-8c0d1e88-4697-4dcd-9f66-5a16657a508f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:21:57.524213 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0d1e88-4697-4dcd-9f66-5a16657a508f admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:21:57.569378 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:21:57.583646 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f5dabc-a08c-41cc-8059-2d944387cfc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:21:57.584285 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f5dabc-a08c-41cc-8059-2d944387cfc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:21:57.584565 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f5dabc-a08c-41cc-8059-2d944387cfc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:21:57.584841 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f5dabc-a08c-41cc-8059-2d944387cfc2 admin admin] PATH_INFO: `/v3/projects/a167bab70f1d4050abc4d9069fb88e3d` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:21:57.585974 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f5dabc-a08c-41cc-8059-2d944387cfc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a167bab70f1d4050abc4d9069fb88e3d)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:21:57.587879 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f5dabc-a08c-41cc-8059-2d944387cfc2 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:21:57.632166 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-96f5dabc-a08c-41cc-8059-2d944387cfc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a167bab70f1d4050abc4d9069fb88e3d', 'request_id': 'req-96f5dabc-a08c-41cc-8059-2d944387cfc2'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:21:57.638712 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-96f5dabc-a08c-41cc-8059-2d944387cfc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a167bab70f1d4050abc4d9069fb88e3d', 'request_id': 'req-96f5dabc-a08c-41cc-8059-2d944387cfc2'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:21:57.645486 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-96f5dabc-a08c-41cc-8059-2d944387cfc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a167bab70f1d4050abc4d9069fb88e3d', 'request_id': 'req-96f5dabc-a08c-41cc-8059-2d944387cfc2'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:21:57.655573 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-96f5dabc-a08c-41cc-8059-2d944387cfc2 admin admin] The token cache is being invalidate because project a167bab70f1d4050abc4d9069fb88e3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 09:21:57.655863 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-96f5dabc-a08c-41cc-8059-2d944387cfc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:21:57.656483 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-96f5dabc-a08c-41cc-8059-2d944387cfc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:21:57.658044 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1193/2371] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 09:21:57 2022] DELETE /identity/v3/projects/a167bab70f1d4050abc4d9069fb88e3d => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:21:57.662847 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd333c3f-d021-4bfa-85cd-0f405a4ece65 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:21:57.663335 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd333c3f-d021-4bfa-85cd-0f405a4ece65 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:21:57.663441 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd333c3f-d021-4bfa-85cd-0f405a4ece65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:21:57.682752 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-dd333c3f-d021-4bfa-85cd-0f405a4ece65 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:21:57.683945 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-dd333c3f-d021-4bfa-85cd-0f405a4ece65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:21:57.742249 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1179/2372] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:21:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:21:57.749933 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626d9173-c400-4e6a-a2b7-eb8e05703f54 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:21:57.750253 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626d9173-c400-4e6a-a2b7-eb8e05703f54 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:21:57.750536 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626d9173-c400-4e6a-a2b7-eb8e05703f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:21:57.770044 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-626d9173-c400-4e6a-a2b7-eb8e05703f54 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:21:57.771251 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-626d9173-c400-4e6a-a2b7-eb8e05703f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:21:57.779593 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1194/2373] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:21:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:21:57.783667 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd333c3f-d021-4bfa-85cd-0f405a4ece65 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:21:57.793357 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd333c3f-d021-4bfa-85cd-0f405a4ece65 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:21:57.798530 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9998dcbb-5baa-4221-a1ab-89b63cecf2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:21:57.799128 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9998dcbb-5baa-4221-a1ab-89b63cecf2c7 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:21:57.799662 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9998dcbb-5baa-4221-a1ab-89b63cecf2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:21:57.799941 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9998dcbb-5baa-4221-a1ab-89b63cecf2c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:21:57.800583 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9998dcbb-5baa-4221-a1ab-89b63cecf2c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 09:21:57.800888 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9998dcbb-5baa-4221-a1ab-89b63cecf2c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:21:57.802541 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9998dcbb-5baa-4221-a1ab-89b63cecf2c7 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:21:57.813938 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1180/2374] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 09:21:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 09:21:57.818017 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626d9173-c400-4e6a-a2b7-eb8e05703f54 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:21:57.827135 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626d9173-c400-4e6a-a2b7-eb8e05703f54 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:21:57.831817 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa27959b-93d2-436e-81da-2ebb2b0d8880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:21:57.832407 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27959b-93d2-436e-81da-2ebb2b0d8880 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:21:57.832692 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27959b-93d2-436e-81da-2ebb2b0d8880 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:21:57.832967 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27959b-93d2-436e-81da-2ebb2b0d8880 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:21:57.833604 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa27959b-93d2-436e-81da-2ebb2b0d8880 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:21:57.835561 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa27959b-93d2-436e-81da-2ebb2b0d8880 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:21:57.863849 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1195/2375] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 09:21:57 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 09:21:57.867609 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9998dcbb-5baa-4221-a1ab-89b63cecf2c7 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:21:57.877398 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9998dcbb-5baa-4221-a1ab-89b63cecf2c7 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:21:57.933034 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45467fcf-756a-4841-a6fd-5ef819010398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:21:57.933573 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45467fcf-756a-4841-a6fd-5ef819010398 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:21:57.933887 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45467fcf-756a-4841-a6fd-5ef819010398 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:21:57.934165 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45467fcf-756a-4841-a6fd-5ef819010398 admin admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:21:57.934931 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45467fcf-756a-4841-a6fd-5ef819010398 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:21:57.936857 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45467fcf-756a-4841-a6fd-5ef819010398 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:21:57.957349 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1181/2376] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 09:21:57 2022] POST /identity/v3/users => generated 292 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:21:57.961463 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa27959b-93d2-436e-81da-2ebb2b0d8880 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:21:57.972512 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa27959b-93d2-436e-81da-2ebb2b0d8880 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:21:57.977176 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df51e39-40d4-4ada-a979-260d1606ef71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:21:57.977811 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df51e39-40d4-4ada-a979-260d1606ef71 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:21:57.978097 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df51e39-40d4-4ada-a979-260d1606ef71 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:21:57.978372 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df51e39-40d4-4ada-a979-260d1606ef71 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:21:57.979010 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df51e39-40d4-4ada-a979-260d1606ef71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:21:57.980610 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df51e39-40d4-4ada-a979-260d1606ef71 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:21:57.990340 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1196/2377] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:21:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:21:57.994326 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45467fcf-756a-4841-a6fd-5ef819010398 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:21:58.004559 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45467fcf-756a-4841-a6fd-5ef819010398 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:21:58.009179 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd33161-6aa8-4125-a463-be7ba4ec24a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:21:58.009816 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd33161-6aa8-4125-a463-be7ba4ec24a5 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:21:58.010100 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd33161-6aa8-4125-a463-be7ba4ec24a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:21:58.010379 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd33161-6aa8-4125-a463-be7ba4ec24a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:21:58.011017 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd33161-6aa8-4125-a463-be7ba4ec24a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:21:58.013114 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd33161-6aa8-4125-a463-be7ba4ec24a5 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:21:58.024103 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1182/2378] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:21:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:21:58.028765 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df51e39-40d4-4ada-a979-260d1606ef71 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:21:58.038713 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df51e39-40d4-4ada-a979-260d1606ef71 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:21:58.043343 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f098ee73-ce49-4a3e-ad48-d97ccf7758fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:21:58.044016 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098ee73-ce49-4a3e-ad48-d97ccf7758fa admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:21:58.044301 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098ee73-ce49-4a3e-ad48-d97ccf7758fa admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:21:58.044585 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098ee73-ce49-4a3e-ad48-d97ccf7758fa admin admin] PATH_INFO: `/v3/projects/d7f660b1d05647c4911bde7b9d9872bf/users/110b6e28b0ec465eb712e255fe583627/roles/3248f0268738489b883041851d681bb7` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:21:58.059439 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f098ee73-ce49-4a3e-ad48-d97ccf7758fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7f660b1d05647c4911bde7b9d9872bf, user_id=110b6e28b0ec465eb712e255fe583627, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:21:58.062136 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f098ee73-ce49-4a3e-ad48-d97ccf7758fa admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:21:58.075737 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1197/2379] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:21:58 2022] PUT /identity/v3/projects/d7f660b1d05647c4911bde7b9d9872bf/users/110b6e28b0ec465eb712e255fe583627/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:21:58.086723 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b50789-a9cc-47e5-a293-cc5be4b00ecd None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:21:58.087011 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b50789-a9cc-47e5-a293-cc5be4b00ecd None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:21:58.087346 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b50789-a9cc-47e5-a293-cc5be4b00ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:21:58.116004 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-f4b50789-a9cc-47e5-a293-cc5be4b00ecd None None] MFA Rules not processed for user `110b6e28b0ec465eb712e255fe583627`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:21:58.117730 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-f4b50789-a9cc-47e5-a293-cc5be4b00ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:21:58.130334 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:21:58.182597 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1183/2380] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:21:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:21:58.187121 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f098ee73-ce49-4a3e-ad48-d97ccf7758fa admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:21:58.247731 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f098ee73-ce49-4a3e-ad48-d97ccf7758fa admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:21:58.252333 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633d445c-71f4-4ce9-9c72-c0ce78f14b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:21:58.252925 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d445c-71f4-4ce9-9c72-c0ce78f14b53 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:21:58.253232 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d445c-71f4-4ce9-9c72-c0ce78f14b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:21:58.253505 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d445c-71f4-4ce9-9c72-c0ce78f14b53 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:21:58.254165 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633d445c-71f4-4ce9-9c72-c0ce78f14b53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:21:58.256050 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633d445c-71f4-4ce9-9c72-c0ce78f14b53 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:21:58.275651 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:21:58.284541 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1198/2381] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 09:21:58 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:21:58.288563 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b50789-a9cc-47e5-a293-cc5be4b00ecd None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:21:58.299050 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b50789-a9cc-47e5-a293-cc5be4b00ecd None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:21:58.356992 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c21ef8d-988c-446e-aa85-d6d658ffa164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:21:58.357709 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c21ef8d-988c-446e-aa85-d6d658ffa164 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:21:58.357907 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c21ef8d-988c-446e-aa85-d6d658ffa164 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:21:58.358181 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c21ef8d-988c-446e-aa85-d6d658ffa164 admin admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:21:58.365876 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c21ef8d-988c-446e-aa85-d6d658ffa164 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:21:58.365876 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c21ef8d-988c-446e-aa85-d6d658ffa164 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:21:58.381524 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1184/2382] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 09:21:58 2022] POST /identity/v3/users => generated 299 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:21:58.389191 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633d445c-71f4-4ce9-9c72-c0ce78f14b53 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:21:58.400115 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633d445c-71f4-4ce9-9c72-c0ce78f14b53 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:21:58.404812 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd997e90-c984-4cb6-a554-eb8b9a384ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:21:58.405418 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd997e90-c984-4cb6-a554-eb8b9a384ab2 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:21:58.405758 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd997e90-c984-4cb6-a554-eb8b9a384ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:21:58.406043 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd997e90-c984-4cb6-a554-eb8b9a384ab2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:21:58.406688 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd997e90-c984-4cb6-a554-eb8b9a384ab2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:21:58.408277 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd997e90-c984-4cb6-a554-eb8b9a384ab2 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:21:58.419085 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1199/2383] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:21:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:21:58.423332 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c21ef8d-988c-446e-aa85-d6d658ffa164 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:21:58.433405 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c21ef8d-988c-446e-aa85-d6d658ffa164 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:21:58.438528 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc379d4-ae1a-45f5-8a5f-d1c5a2dbd2ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:21:58.439108 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc379d4-ae1a-45f5-8a5f-d1c5a2dbd2ff admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:21:58.439387 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc379d4-ae1a-45f5-8a5f-d1c5a2dbd2ff admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:21:58.439660 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc379d4-ae1a-45f5-8a5f-d1c5a2dbd2ff admin admin] PATH_INFO: `/v3/projects/69c5d14cfcb841948491f78ccfdbb2b9/users/e34eccc17a65418abc33e9152c513484/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:21:58.454521 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc379d4-ae1a-45f5-8a5f-d1c5a2dbd2ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=69c5d14cfcb841948491f78ccfdbb2b9, user_id=e34eccc17a65418abc33e9152c513484, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:21:58.457160 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc379d4-ae1a-45f5-8a5f-d1c5a2dbd2ff admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:21:58.469479 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1185/2384] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:21:58 2022] PUT /identity/v3/projects/69c5d14cfcb841948491f78ccfdbb2b9/users/e34eccc17a65418abc33e9152c513484/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:21:58.473350 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd997e90-c984-4cb6-a554-eb8b9a384ab2 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:21:58.483479 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd997e90-c984-4cb6-a554-eb8b9a384ab2 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:21:58.540231 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b3e7a7-bf4d-432e-9d69-eb6a3e7b0bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:21:58.540827 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b3e7a7-bf4d-432e-9d69-eb6a3e7b0bcb admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:21:58.541103 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b3e7a7-bf4d-432e-9d69-eb6a3e7b0bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:21:58.541370 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b3e7a7-bf4d-432e-9d69-eb6a3e7b0bcb admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:21:58.542033 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b3e7a7-bf4d-432e-9d69-eb6a3e7b0bcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:21:58.543651 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b3e7a7-bf4d-432e-9d69-eb6a3e7b0bcb admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:21:58.553280 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1200/2385] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:21:58 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:21:58.557426 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc379d4-ae1a-45f5-8a5f-d1c5a2dbd2ff admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:21:58.568233 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc379d4-ae1a-45f5-8a5f-d1c5a2dbd2ff admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:21:58.572827 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d1eb7e-2176-4764-bb3c-979846854ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:21:58.573467 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d1eb7e-2176-4764-bb3c-979846854ee4 admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:21:58.573779 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d1eb7e-2176-4764-bb3c-979846854ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:21:58.574056 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d1eb7e-2176-4764-bb3c-979846854ee4 admin admin] PATH_INFO: `/v3/projects/69c5d14cfcb841948491f78ccfdbb2b9/users/e34eccc17a65418abc33e9152c513484/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:21:58.576556 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d1eb7e-2176-4764-bb3c-979846854ee4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69c5d14cfcb841948491f78ccfdbb2b9, user_id=e34eccc17a65418abc33e9152c513484, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:21:58.579217 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d1eb7e-2176-4764-bb3c-979846854ee4 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:21:58.589186 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1186/2386] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:21:58 2022] PUT /identity/v3/projects/69c5d14cfcb841948491f78ccfdbb2b9/users/e34eccc17a65418abc33e9152c513484/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:21:58.598839 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d7915e-5fb0-4dbd-bf47-8231dcc3cf79 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:21:58.599306 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d7915e-5fb0-4dbd-bf47-8231dcc3cf79 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:21:58.599421 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d7915e-5fb0-4dbd-bf47-8231dcc3cf79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:21:58.627135 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-d4d7915e-5fb0-4dbd-bf47-8231dcc3cf79 None None] MFA Rules not processed for user `e34eccc17a65418abc33e9152c513484`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:21:58.628349 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-d4d7915e-5fb0-4dbd-bf47-8231dcc3cf79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:21:58.696190 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1201/2387] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:21:58 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:21:58.707108 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d1eb7e-2176-4764-bb3c-979846854ee4 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:21:58.708772 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-34d1eb7e-2176-4764-bb3c-979846854ee4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:21:58.765490 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d1eb7e-2176-4764-bb3c-979846854ee4 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:21:58.769063 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd88e66-09a1-4e3d-b282-defaa2c0f4e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:21:58.769657 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd88e66-09a1-4e3d-b282-defaa2c0f4e8 service cinder] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:21:58.769963 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd88e66-09a1-4e3d-b282-defaa2c0f4e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:21:58.770232 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd88e66-09a1-4e3d-b282-defaa2c0f4e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:21:58.789215 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:21:58.825956 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd88e66-09a1-4e3d-b282-defaa2c0f4e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:21:58.827376 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd88e66-09a1-4e3d-b282-defaa2c0f4e8 service cinder] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:21:58.836205 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1187/2388] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:21:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:22:03.020097 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d7915e-5fb0-4dbd-bf47-8231dcc3cf79 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:22:03.021468 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-d4d7915e-5fb0-4dbd-bf47-8231dcc3cf79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:22:03.033200 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d7915e-5fb0-4dbd-bf47-8231dcc3cf79 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:22:03.036691 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e98d4e-b6d2-4de3-a1b4-4cae3188147e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:22:03.037280 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e98d4e-b6d2-4de3-a1b4-4cae3188147e service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:03.037556 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e98d4e-b6d2-4de3-a1b4-4cae3188147e service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:03.037858 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e98d4e-b6d2-4de3-a1b4-4cae3188147e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:03.046538 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e98d4e-b6d2-4de3-a1b4-4cae3188147e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:22:03.047936 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e98d4e-b6d2-4de3-a1b4-4cae3188147e service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:22:03.057308 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1202/2389] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:22:03 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:22:07.943221 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd88e66-09a1-4e3d-b282-defaa2c0f4e8 service cinder] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:22:08.003641 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd88e66-09a1-4e3d-b282-defaa2c0f4e8 service cinder] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:22:08.008094 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cafb4dc-7726-40ab-94cd-4e255f717d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:22:08.008715 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cafb4dc-7726-40ab-94cd-4e255f717d4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:08.008996 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cafb4dc-7726-40ab-94cd-4e255f717d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:08.009269 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cafb4dc-7726-40ab-94cd-4e255f717d4b admin admin] PATH_INFO: `/v3/users/110b6e28b0ec465eb712e255fe583627` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:08.010393 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cafb4dc-7726-40ab-94cd-4e255f717d4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=110b6e28b0ec465eb712e255fe583627)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:22:08.012365 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cafb4dc-7726-40ab-94cd-4e255f717d4b admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:22:08.061238 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-8cafb4dc-7726-40ab-94cd-4e255f717d4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '110b6e28b0ec465eb712e255fe583627'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:22:08.061920 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-8cafb4dc-7726-40ab-94cd-4e255f717d4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '110b6e28b0ec465eb712e255fe583627'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:22:08.073708 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-8cafb4dc-7726-40ab-94cd-4e255f717d4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '110b6e28b0ec465eb712e255fe583627'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:22:08.080808 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-8cafb4dc-7726-40ab-94cd-4e255f717d4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '110b6e28b0ec465eb712e255fe583627'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:22:08.081428 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-8cafb4dc-7726-40ab-94cd-4e255f717d4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '110b6e28b0ec465eb712e255fe583627'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:22:08.094667 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1188/2390] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 09:22:07 2022] DELETE /identity/v3/users/110b6e28b0ec465eb712e255fe583627 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:22:08.099102 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610eff8c-0fca-48a4-906c-146a0c878622 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:08.099598 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610eff8c-0fca-48a4-906c-146a0c878622 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:08.099700 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610eff8c-0fca-48a4-906c-146a0c878622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:08.118830 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-610eff8c-0fca-48a4-906c-146a0c878622 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:22:08.119962 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-610eff8c-0fca-48a4-906c-146a0c878622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:22:08.165956 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:22:08.178102 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1203/2391] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:22:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 09:22:08.187846 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cafb4dc-7726-40ab-94cd-4e255f717d4b admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:22:08.189305 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-8cafb4dc-7726-40ab-94cd-4e255f717d4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:22:08.234626 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:22:08.248084 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cafb4dc-7726-40ab-94cd-4e255f717d4b admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:22:08.251676 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b122686-956d-4dd6-bef6-23bd687adc32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:22:08.252272 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b122686-956d-4dd6-bef6-23bd687adc32 service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:08.252574 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b122686-956d-4dd6-bef6-23bd687adc32 service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:08.252848 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b122686-956d-4dd6-bef6-23bd687adc32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:08.261686 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b122686-956d-4dd6-bef6-23bd687adc32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:22:08.263114 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b122686-956d-4dd6-bef6-23bd687adc32 service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:22:08.272409 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1189/2392] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:22:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:22:08.534467 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610eff8c-0fca-48a4-906c-146a0c878622 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:22:08.535979 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-610eff8c-0fca-48a4-906c-146a0c878622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:22:08.548065 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610eff8c-0fca-48a4-906c-146a0c878622 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:22:08.553255 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180a7f2d-4211-481d-b325-758ad08c6b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:22:08.553919 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a7f2d-4211-481d-b325-758ad08c6b5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:08.554210 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a7f2d-4211-481d-b325-758ad08c6b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:08.554488 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a7f2d-4211-481d-b325-758ad08c6b5e admin admin] PATH_INFO: `/v3/projects/d7f660b1d05647c4911bde7b9d9872bf` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:08.555634 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180a7f2d-4211-481d-b325-758ad08c6b5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7f660b1d05647c4911bde7b9d9872bf)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:22:08.557617 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180a7f2d-4211-481d-b325-758ad08c6b5e admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:22:08.606866 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-180a7f2d-4211-481d-b325-758ad08c6b5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7f660b1d05647c4911bde7b9d9872bf', 'request_id': 'req-180a7f2d-4211-481d-b325-758ad08c6b5e'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:22:08.613272 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-180a7f2d-4211-481d-b325-758ad08c6b5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7f660b1d05647c4911bde7b9d9872bf', 'request_id': 'req-180a7f2d-4211-481d-b325-758ad08c6b5e'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:22:08.620661 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-180a7f2d-4211-481d-b325-758ad08c6b5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7f660b1d05647c4911bde7b9d9872bf', 'request_id': 'req-180a7f2d-4211-481d-b325-758ad08c6b5e'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:22:08.630938 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-180a7f2d-4211-481d-b325-758ad08c6b5e admin admin] The token cache is being invalidate because project d7f660b1d05647c4911bde7b9d9872bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 09:22:08.631222 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-180a7f2d-4211-481d-b325-758ad08c6b5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:22:08.631840 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-180a7f2d-4211-481d-b325-758ad08c6b5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:22:08.633326 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1204/2393] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 09:22:08 2022] DELETE /identity/v3/projects/d7f660b1d05647c4911bde7b9d9872bf => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:22:08.637268 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b122686-956d-4dd6-bef6-23bd687adc32 service neutron] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:22:08.647361 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b122686-956d-4dd6-bef6-23bd687adc32 service neutron] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:22:08.703189 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801dca1f-d67d-428b-9863-82579b277cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:22:08.703763 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801dca1f-d67d-428b-9863-82579b277cba admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:08.704140 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801dca1f-d67d-428b-9863-82579b277cba admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:08.704419 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801dca1f-d67d-428b-9863-82579b277cba admin admin] PATH_INFO: `/v3/users/e34eccc17a65418abc33e9152c513484` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:08.705547 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801dca1f-d67d-428b-9863-82579b277cba admin admin] RBAC: Authorizing `identity:delete_user(user_id=e34eccc17a65418abc33e9152c513484)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:22:08.707527 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801dca1f-d67d-428b-9863-82579b277cba admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:22:08.752486 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-801dca1f-d67d-428b-9863-82579b277cba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e34eccc17a65418abc33e9152c513484'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:22:08.753131 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-801dca1f-d67d-428b-9863-82579b277cba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e34eccc17a65418abc33e9152c513484'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:22:08.765462 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-801dca1f-d67d-428b-9863-82579b277cba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e34eccc17a65418abc33e9152c513484'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:22:08.772479 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-801dca1f-d67d-428b-9863-82579b277cba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e34eccc17a65418abc33e9152c513484'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:22:08.773094 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-801dca1f-d67d-428b-9863-82579b277cba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e34eccc17a65418abc33e9152c513484'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:22:08.785137 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1190/2394] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 09:22:08 2022] DELETE /identity/v3/users/e34eccc17a65418abc33e9152c513484 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:22:09.057017 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180a7f2d-4211-481d-b325-758ad08c6b5e admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:22:09.058574 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-180a7f2d-4211-481d-b325-758ad08c6b5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:22:09.072054 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180a7f2d-4211-481d-b325-758ad08c6b5e admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:22:09.128638 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe3266c-0520-408a-b294-c5311c709b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:22:09.129347 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe3266c-0520-408a-b294-c5311c709b6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:09.129541 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe3266c-0520-408a-b294-c5311c709b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:09.129854 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe3266c-0520-408a-b294-c5311c709b6f admin admin] PATH_INFO: `/v3/projects/69c5d14cfcb841948491f78ccfdbb2b9` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:09.130984 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe3266c-0520-408a-b294-c5311c709b6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=69c5d14cfcb841948491f78ccfdbb2b9)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:22:09.132882 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe3266c-0520-408a-b294-c5311c709b6f admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:22:09.178365 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-6fe3266c-0520-408a-b294-c5311c709b6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69c5d14cfcb841948491f78ccfdbb2b9', 'request_id': 'req-6fe3266c-0520-408a-b294-c5311c709b6f'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:22:09.184568 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-6fe3266c-0520-408a-b294-c5311c709b6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69c5d14cfcb841948491f78ccfdbb2b9', 'request_id': 'req-6fe3266c-0520-408a-b294-c5311c709b6f'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:22:09.191477 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-6fe3266c-0520-408a-b294-c5311c709b6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69c5d14cfcb841948491f78ccfdbb2b9', 'request_id': 'req-6fe3266c-0520-408a-b294-c5311c709b6f'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:22:09.201875 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-6fe3266c-0520-408a-b294-c5311c709b6f admin admin] The token cache is being invalidate because project 69c5d14cfcb841948491f78ccfdbb2b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 09:22:09.202167 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-6fe3266c-0520-408a-b294-c5311c709b6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:22:09.202835 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-6fe3266c-0520-408a-b294-c5311c709b6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:22:09.204769 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1205/2395] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 09:22:09 2022] DELETE /identity/v3/projects/69c5d14cfcb841948491f78ccfdbb2b9 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:22:09.210866 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10ee2b9-ada5-4a45-8c3c-f3f14b760fb3 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:09.211342 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10ee2b9-ada5-4a45-8c3c-f3f14b760fb3 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:09.211342 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10ee2b9-ada5-4a45-8c3c-f3f14b760fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:09.231669 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-f10ee2b9-ada5-4a45-8c3c-f3f14b760fb3 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:22:09.232879 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-f10ee2b9-ada5-4a45-8c3c-f3f14b760fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:22:09.290511 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1191/2396] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:22:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:22:09.300403 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a74016-a4f4-40b2-8db0-0fb0312468d8 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:09.300712 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a74016-a4f4-40b2-8db0-0fb0312468d8 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:09.301013 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a74016-a4f4-40b2-8db0-0fb0312468d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:09.322736 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-77a74016-a4f4-40b2-8db0-0fb0312468d8 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:22:09.323933 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-77a74016-a4f4-40b2-8db0-0fb0312468d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:22:09.331735 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1206/2397] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:22:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:22:09.335944 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10ee2b9-ada5-4a45-8c3c-f3f14b760fb3 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:22:09.341892 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:22:09.346103 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10ee2b9-ada5-4a45-8c3c-f3f14b760fb3 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:22:09.352166 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c22461a-ca0d-453e-a3c7-bb2cbf785b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:22:09.352762 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c22461a-ca0d-453e-a3c7-bb2cbf785b29 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:09.353043 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c22461a-ca0d-453e-a3c7-bb2cbf785b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:09.353318 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c22461a-ca0d-453e-a3c7-bb2cbf785b29 admin admin] PATH_INFO: `/v3/domains` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:09.353990 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c22461a-ca0d-453e-a3c7-bb2cbf785b29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 09:22:09.354296 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c22461a-ca0d-453e-a3c7-bb2cbf785b29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:22:09.355892 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c22461a-ca0d-453e-a3c7-bb2cbf785b29 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:22:09.367732 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1192/2398] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 09:22:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 09:22:09.371980 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a74016-a4f4-40b2-8db0-0fb0312468d8 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:22:09.376948 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:22:09.383428 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a74016-a4f4-40b2-8db0-0fb0312468d8 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:22:09.389154 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122d67d9-6202-436d-8761-69ba0a064978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:22:09.389808 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122d67d9-6202-436d-8761-69ba0a064978 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:09.390091 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122d67d9-6202-436d-8761-69ba0a064978 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:09.390362 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122d67d9-6202-436d-8761-69ba0a064978 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:09.391000 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122d67d9-6202-436d-8761-69ba0a064978 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:22:09.392884 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122d67d9-6202-436d-8761-69ba0a064978 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:22:09.419132 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1207/2399] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 09:22:09 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:22:09.423779 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c22461a-ca0d-453e-a3c7-bb2cbf785b29 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:22:09.434145 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c22461a-ca0d-453e-a3c7-bb2cbf785b29 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:22:09.489346 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e05e4c-7142-4041-b0d4-0baf19227382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:22:09.489913 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e05e4c-7142-4041-b0d4-0baf19227382 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:09.490198 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e05e4c-7142-4041-b0d4-0baf19227382 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:09.490481 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e05e4c-7142-4041-b0d4-0baf19227382 admin admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:09.491108 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e05e4c-7142-4041-b0d4-0baf19227382 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:22:09.493028 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e05e4c-7142-4041-b0d4-0baf19227382 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:22:09.514673 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1193/2400] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 09:22:09 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:22:09.519021 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122d67d9-6202-436d-8761-69ba0a064978 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:22:09.529320 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122d67d9-6202-436d-8761-69ba0a064978 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:22:09.534527 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2bce55-4bf3-4a85-bd4b-b19b18fb6144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:22:09.535114 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2bce55-4bf3-4a85-bd4b-b19b18fb6144 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:09.535391 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2bce55-4bf3-4a85-bd4b-b19b18fb6144 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:09.535670 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2bce55-4bf3-4a85-bd4b-b19b18fb6144 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:09.536302 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2bce55-4bf3-4a85-bd4b-b19b18fb6144 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:22:09.537922 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2bce55-4bf3-4a85-bd4b-b19b18fb6144 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:22:09.548973 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1208/2401] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:22:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:22:09.553039 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e05e4c-7142-4041-b0d4-0baf19227382 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:22:09.562821 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e05e4c-7142-4041-b0d4-0baf19227382 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:22:09.567345 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa28169f-5f67-4496-b029-39a18b7e7a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:22:09.567931 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa28169f-5f67-4496-b029-39a18b7e7a52 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:09.568213 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa28169f-5f67-4496-b029-39a18b7e7a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:09.568487 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa28169f-5f67-4496-b029-39a18b7e7a52 admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:09.569121 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa28169f-5f67-4496-b029-39a18b7e7a52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:22:09.570757 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa28169f-5f67-4496-b029-39a18b7e7a52 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:22:09.581248 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1194/2402] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:22:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:22:09.585068 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2bce55-4bf3-4a85-bd4b-b19b18fb6144 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:22:09.595865 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2bce55-4bf3-4a85-bd4b-b19b18fb6144 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:22:09.600510 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a91e3b-3625-47ef-b2dc-9462dfddc2c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:22:09.601153 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a91e3b-3625-47ef-b2dc-9462dfddc2c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:09.601432 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a91e3b-3625-47ef-b2dc-9462dfddc2c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:09.601764 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a91e3b-3625-47ef-b2dc-9462dfddc2c4 admin admin] PATH_INFO: `/v3/projects/37b1061b84ec42fda73f6c0779d565d3/users/5ddee3d55653421c8a4110c97053a431/roles/3248f0268738489b883041851d681bb7` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:09.616565 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a91e3b-3625-47ef-b2dc-9462dfddc2c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37b1061b84ec42fda73f6c0779d565d3, user_id=5ddee3d55653421c8a4110c97053a431, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:22:09.619223 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a91e3b-3625-47ef-b2dc-9462dfddc2c4 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:22:09.631651 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1209/2403] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:22:09 2022] PUT /identity/v3/projects/37b1061b84ec42fda73f6c0779d565d3/users/5ddee3d55653421c8a4110c97053a431/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 09:22:09.642793 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb9d22-4956-450c-8834-4c435384be22 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:09.643065 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb9d22-4956-450c-8834-4c435384be22 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:09.643337 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb9d22-4956-450c-8834-4c435384be22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:09.671626 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-52fb9d22-4956-450c-8834-4c435384be22 None None] MFA Rules not processed for user `5ddee3d55653421c8a4110c97053a431`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:22:09.672824 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-52fb9d22-4956-450c-8834-4c435384be22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:22:09.736532 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1195/2404] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:22:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:22:09.740965 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a91e3b-3625-47ef-b2dc-9462dfddc2c4 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:22:09.800117 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a91e3b-3625-47ef-b2dc-9462dfddc2c4 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:22:09.804637 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4a274e-9650-46e4-9c52-49a07407bb49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:22:09.805226 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4a274e-9650-46e4-9c52-49a07407bb49 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:09.805505 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4a274e-9650-46e4-9c52-49a07407bb49 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:09.805809 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4a274e-9650-46e4-9c52-49a07407bb49 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:09.806428 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4a274e-9650-46e4-9c52-49a07407bb49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:22:09.808316 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4a274e-9650-46e4-9c52-49a07407bb49 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:22:09.833540 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1210/2405] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 09:22:09 2022] POST /identity/v3/projects => generated 385 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:22:09.837572 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fb9d22-4956-450c-8834-4c435384be22 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:22:09.848686 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fb9d22-4956-450c-8834-4c435384be22 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:22:09.903631 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98d1114-eca8-4160-98ec-4b79dc70fd39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:22:09.904232 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d1114-eca8-4160-98ec-4b79dc70fd39 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:09.904513 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d1114-eca8-4160-98ec-4b79dc70fd39 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:09.904783 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d1114-eca8-4160-98ec-4b79dc70fd39 admin admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:09.906019 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98d1114-eca8-4160-98ec-4b79dc70fd39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:22:09.907977 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98d1114-eca8-4160-98ec-4b79dc70fd39 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:22:09.915719 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:22:09.928965 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1196/2406] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 09:22:09 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:22:09.932840 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4a274e-9650-46e4-9c52-49a07407bb49 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:22:09.943264 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4a274e-9650-46e4-9c52-49a07407bb49 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:22:09.948854 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5b4f27-88cb-4f41-8861-435fa75dcf14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:22:09.949447 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b4f27-88cb-4f41-8861-435fa75dcf14 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:09.949759 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b4f27-88cb-4f41-8861-435fa75dcf14 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:09.950040 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b4f27-88cb-4f41-8861-435fa75dcf14 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:09.950675 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5b4f27-88cb-4f41-8861-435fa75dcf14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:22:09.952804 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5b4f27-88cb-4f41-8861-435fa75dcf14 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:22:09.963109 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1211/2407] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:22:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:22:09.966835 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98d1114-eca8-4160-98ec-4b79dc70fd39 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:22:09.977309 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98d1114-eca8-4160-98ec-4b79dc70fd39 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:22:09.981932 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b65d443-9cec-49a8-8fdd-bf76df0915ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:22:09.982518 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65d443-9cec-49a8-8fdd-bf76df0915ff admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:09.982805 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65d443-9cec-49a8-8fdd-bf76df0915ff admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:09.983090 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65d443-9cec-49a8-8fdd-bf76df0915ff admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:09.983732 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b65d443-9cec-49a8-8fdd-bf76df0915ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:22:09.985345 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b65d443-9cec-49a8-8fdd-bf76df0915ff admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:22:09.998731 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1197/2408] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:22:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:22:09.999854 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5b4f27-88cb-4f41-8861-435fa75dcf14 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:22:10.009934 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5b4f27-88cb-4f41-8861-435fa75dcf14 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:22:10.015189 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4f80a6-293b-4bf9-9b51-85605eba3d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:22:10.015833 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4f80a6-293b-4bf9-9b51-85605eba3d05 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:10.016118 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4f80a6-293b-4bf9-9b51-85605eba3d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:10.016455 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4f80a6-293b-4bf9-9b51-85605eba3d05 admin admin] PATH_INFO: `/v3/projects/2e480e7e000748abb7fa95be606fd0c2/users/4c28652a6b344bf283da93fc8ef45fba/roles/3248f0268738489b883041851d681bb7` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:10.031916 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4f80a6-293b-4bf9-9b51-85605eba3d05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e480e7e000748abb7fa95be606fd0c2, user_id=4c28652a6b344bf283da93fc8ef45fba, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:22:10.034674 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4f80a6-293b-4bf9-9b51-85605eba3d05 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:22:10.047837 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1212/2409] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:22:09 2022] PUT /identity/v3/projects/2e480e7e000748abb7fa95be606fd0c2/users/4c28652a6b344bf283da93fc8ef45fba/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:22:10.058341 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae16fc-ca57-4be5-91ec-159b25966d33 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:10.058629 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae16fc-ca57-4be5-91ec-159b25966d33 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:10.058908 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae16fc-ca57-4be5-91ec-159b25966d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:10.086661 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-b3ae16fc-ca57-4be5-91ec-159b25966d33 None None] MFA Rules not processed for user `4c28652a6b344bf283da93fc8ef45fba`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:22:10.087896 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-b3ae16fc-ca57-4be5-91ec-159b25966d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:22:10.150936 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1198/2410] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:22:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:22:10.155881 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4f80a6-293b-4bf9-9b51-85605eba3d05 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:22:10.211564 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:22:10.217468 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4f80a6-293b-4bf9-9b51-85605eba3d05 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:22:10.223178 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc15dfd-a805-4d53-9b7d-f8753f1f6041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:22:10.223786 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc15dfd-a805-4d53-9b7d-f8753f1f6041 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:10.224096 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc15dfd-a805-4d53-9b7d-f8753f1f6041 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:10.224376 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc15dfd-a805-4d53-9b7d-f8753f1f6041 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:10.225004 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc15dfd-a805-4d53-9b7d-f8753f1f6041 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:22:10.226940 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc15dfd-a805-4d53-9b7d-f8753f1f6041 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:22:10.253057 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1213/2411] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 09:22:10 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:22:10.257359 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ae16fc-ca57-4be5-91ec-159b25966d33 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:22:10.268553 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ae16fc-ca57-4be5-91ec-159b25966d33 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:22:10.323028 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd558c2e-517b-489b-8589-01b0647d3457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:22:10.323624 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd558c2e-517b-489b-8589-01b0647d3457 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:10.323900 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd558c2e-517b-489b-8589-01b0647d3457 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:10.324171 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd558c2e-517b-489b-8589-01b0647d3457 admin admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:10.324796 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd558c2e-517b-489b-8589-01b0647d3457 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:22:10.326759 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd558c2e-517b-489b-8589-01b0647d3457 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:22:10.347581 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1199/2412] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 09:22:10 2022] POST /identity/v3/users => generated 299 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:22:10.351595 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc15dfd-a805-4d53-9b7d-f8753f1f6041 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:22:10.361571 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc15dfd-a805-4d53-9b7d-f8753f1f6041 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:22:10.366207 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aebc0c2-e174-45ed-8f27-73fd2af8688f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:22:10.366792 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aebc0c2-e174-45ed-8f27-73fd2af8688f admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:10.367070 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aebc0c2-e174-45ed-8f27-73fd2af8688f admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:10.367344 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aebc0c2-e174-45ed-8f27-73fd2af8688f admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:10.367971 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aebc0c2-e174-45ed-8f27-73fd2af8688f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:22:10.369550 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aebc0c2-e174-45ed-8f27-73fd2af8688f admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:22:10.380333 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1214/2413] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:22:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:22:10.384375 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd558c2e-517b-489b-8589-01b0647d3457 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:22:10.395006 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd558c2e-517b-489b-8589-01b0647d3457 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:22:10.400015 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796684ef-68a6-4967-b85f-82bbf6146457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:22:10.400689 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796684ef-68a6-4967-b85f-82bbf6146457 admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:10.400948 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796684ef-68a6-4967-b85f-82bbf6146457 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:10.401255 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796684ef-68a6-4967-b85f-82bbf6146457 admin admin] PATH_INFO: `/v3/projects/503624d95059472d8dd8172dd393c1f3/users/f327ca6ca1d34a0caa8f4a5041d1080c/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:10.416120 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796684ef-68a6-4967-b85f-82bbf6146457 admin admin] RBAC: Authorizing `identity:create_grant(project_id=503624d95059472d8dd8172dd393c1f3, user_id=f327ca6ca1d34a0caa8f4a5041d1080c, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:22:10.418817 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796684ef-68a6-4967-b85f-82bbf6146457 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:22:10.431445 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1200/2414] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:22:10 2022] PUT /identity/v3/projects/503624d95059472d8dd8172dd393c1f3/users/f327ca6ca1d34a0caa8f4a5041d1080c/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:22:10.435354 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aebc0c2-e174-45ed-8f27-73fd2af8688f admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:22:10.445595 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aebc0c2-e174-45ed-8f27-73fd2af8688f admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:22:10.500175 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2442b33b-5a99-445e-9866-3bccc8e38604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:22:10.500768 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2442b33b-5a99-445e-9866-3bccc8e38604 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:10.501051 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2442b33b-5a99-445e-9866-3bccc8e38604 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:10.501327 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2442b33b-5a99-445e-9866-3bccc8e38604 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:10.501999 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2442b33b-5a99-445e-9866-3bccc8e38604 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:22:10.503630 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2442b33b-5a99-445e-9866-3bccc8e38604 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:22:10.512947 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1215/2415] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:22:10 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:22:10.517667 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796684ef-68a6-4967-b85f-82bbf6146457 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:22:10.527659 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796684ef-68a6-4967-b85f-82bbf6146457 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:22:10.532182 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a52766-bf3c-4f38-90cb-e2196547cb40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:22:10.532815 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a52766-bf3c-4f38-90cb-e2196547cb40 admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:10.533093 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a52766-bf3c-4f38-90cb-e2196547cb40 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:10.533361 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a52766-bf3c-4f38-90cb-e2196547cb40 admin admin] PATH_INFO: `/v3/projects/503624d95059472d8dd8172dd393c1f3/users/f327ca6ca1d34a0caa8f4a5041d1080c/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:10.535283 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a52766-bf3c-4f38-90cb-e2196547cb40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=503624d95059472d8dd8172dd393c1f3, user_id=f327ca6ca1d34a0caa8f4a5041d1080c, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:22:10.537952 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a52766-bf3c-4f38-90cb-e2196547cb40 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:22:10.547770 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1201/2416] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:22:10 2022] PUT /identity/v3/projects/503624d95059472d8dd8172dd393c1f3/users/f327ca6ca1d34a0caa8f4a5041d1080c/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:22:10.558891 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd142534-3dd1-41cd-bada-81683a517f8c None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:10.559182 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd142534-3dd1-41cd-bada-81683a517f8c None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:10.559487 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd142534-3dd1-41cd-bada-81683a517f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:10.587680 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-bd142534-3dd1-41cd-bada-81683a517f8c None None] MFA Rules not processed for user `f327ca6ca1d34a0caa8f4a5041d1080c`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:22:10.588897 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-bd142534-3dd1-41cd-bada-81683a517f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:22:10.655673 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1216/2417] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:22:10 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:22:10.667626 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a52766-bf3c-4f38-90cb-e2196547cb40 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:22:10.669322 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-93a52766-bf3c-4f38-90cb-e2196547cb40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:22:10.692445 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:22:10.728457 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a52766-bf3c-4f38-90cb-e2196547cb40 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:22:10.732074 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6de215-f821-4d44-8f04-e114fb69094b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:22:10.732669 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6de215-f821-4d44-8f04-e114fb69094b service cinder] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:10.732948 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6de215-f821-4d44-8f04-e114fb69094b service cinder] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:10.733225 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6de215-f821-4d44-8f04-e114fb69094b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:10.787751 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6de215-f821-4d44-8f04-e114fb69094b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:22:10.789115 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6de215-f821-4d44-8f04-e114fb69094b service cinder] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:22:10.797655 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1202/2418] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:22:10 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:22:12.288258 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd142534-3dd1-41cd-bada-81683a517f8c None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:22:12.289628 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-bd142534-3dd1-41cd-bada-81683a517f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:22:12.302198 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd142534-3dd1-41cd-bada-81683a517f8c None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:22:12.305763 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14661f9-483c-4e58-8d24-d8deb5ed6827 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:22:12.306386 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14661f9-483c-4e58-8d24-d8deb5ed6827 service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:12.306686 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14661f9-483c-4e58-8d24-d8deb5ed6827 service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:12.306984 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14661f9-483c-4e58-8d24-d8deb5ed6827 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:12.354658 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:22:12.365838 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14661f9-483c-4e58-8d24-d8deb5ed6827 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:22:12.367398 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14661f9-483c-4e58-8d24-d8deb5ed6827 service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:22:12.376708 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1217/2419] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:22:12 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:22:12.877950 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6de215-f821-4d44-8f04-e114fb69094b service cinder] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:22:12.886576 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6de215-f821-4d44-8f04-e114fb69094b service cinder] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:22:12.890785 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f3d325-2911-4515-8243-7e023cc2ed36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:22:12.891381 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f3d325-2911-4515-8243-7e023cc2ed36 service cinder] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:12.891660 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f3d325-2911-4515-8243-7e023cc2ed36 service cinder] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:12.891936 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f3d325-2911-4515-8243-7e023cc2ed36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:12.900656 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f3d325-2911-4515-8243-7e023cc2ed36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:22:12.902114 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f3d325-2911-4515-8243-7e023cc2ed36 service cinder] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:22:12.911407 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1203/2420] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:22:12 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:22:16.761465 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14661f9-483c-4e58-8d24-d8deb5ed6827 service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:22:16.825662 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14661f9-483c-4e58-8d24-d8deb5ed6827 service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:22:16.830780 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e1a8c-99af-4cd6-af9c-5eb7bc57345b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:22:16.831437 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e1a8c-99af-4cd6-af9c-5eb7bc57345b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:16.831681 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e1a8c-99af-4cd6-af9c-5eb7bc57345b admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:16.831956 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e1a8c-99af-4cd6-af9c-5eb7bc57345b admin admin] PATH_INFO: `/v3/users/5ddee3d55653421c8a4110c97053a431` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:16.833060 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64e1a8c-99af-4cd6-af9c-5eb7bc57345b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ddee3d55653421c8a4110c97053a431)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:22:16.835011 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64e1a8c-99af-4cd6-af9c-5eb7bc57345b admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:22:16.881740 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d64e1a8c-99af-4cd6-af9c-5eb7bc57345b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ddee3d55653421c8a4110c97053a431'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:22:16.882399 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d64e1a8c-99af-4cd6-af9c-5eb7bc57345b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ddee3d55653421c8a4110c97053a431'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:22:16.890037 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d64e1a8c-99af-4cd6-af9c-5eb7bc57345b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ddee3d55653421c8a4110c97053a431'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:22:16.901587 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d64e1a8c-99af-4cd6-af9c-5eb7bc57345b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ddee3d55653421c8a4110c97053a431'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:22:16.910054 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d64e1a8c-99af-4cd6-af9c-5eb7bc57345b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ddee3d55653421c8a4110c97053a431'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:22:16.916706 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1218/2421] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 09:22:16 2022] DELETE /identity/v3/users/5ddee3d55653421c8a4110c97053a431 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:22:16.921310 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7689029-a512-4c1b-a9fa-faab77fa93c3 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:16.921821 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7689029-a512-4c1b-a9fa-faab77fa93c3 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:16.921940 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7689029-a512-4c1b-a9fa-faab77fa93c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:16.941181 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-a7689029-a512-4c1b-a9fa-faab77fa93c3 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:22:16.942350 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-a7689029-a512-4c1b-a9fa-faab77fa93c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:22:17.002658 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1204/2422] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:22:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:22:17.012230 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e1a8c-99af-4cd6-af9c-5eb7bc57345b admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:22:17.013655 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-d64e1a8c-99af-4cd6-af9c-5eb7bc57345b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:22:17.072374 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e1a8c-99af-4cd6-af9c-5eb7bc57345b admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:22:17.076034 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd75d352-836a-471e-aece-4547b67a4f84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:22:17.076653 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd75d352-836a-471e-aece-4547b67a4f84 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:17.076965 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd75d352-836a-471e-aece-4547b67a4f84 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:17.077242 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd75d352-836a-471e-aece-4547b67a4f84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:17.086375 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd75d352-836a-471e-aece-4547b67a4f84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:22:17.087834 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd75d352-836a-471e-aece-4547b67a4f84 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:22:17.098740 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1219/2423] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:22:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:22:17.365072 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7689029-a512-4c1b-a9fa-faab77fa93c3 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:22:17.366523 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-a7689029-a512-4c1b-a9fa-faab77fa93c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:22:17.380581 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7689029-a512-4c1b-a9fa-faab77fa93c3 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:22:17.385303 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96631267-9adc-4ffd-a9c3-01e8e3d49b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:22:17.385981 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96631267-9adc-4ffd-a9c3-01e8e3d49b17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:17.386264 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96631267-9adc-4ffd-a9c3-01e8e3d49b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:17.386545 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96631267-9adc-4ffd-a9c3-01e8e3d49b17 admin admin] PATH_INFO: `/v3/projects/37b1061b84ec42fda73f6c0779d565d3` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:17.388408 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96631267-9adc-4ffd-a9c3-01e8e3d49b17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37b1061b84ec42fda73f6c0779d565d3)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:22:17.391046 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96631267-9adc-4ffd-a9c3-01e8e3d49b17 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:22:17.439734 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-96631267-9adc-4ffd-a9c3-01e8e3d49b17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37b1061b84ec42fda73f6c0779d565d3', 'request_id': 'req-96631267-9adc-4ffd-a9c3-01e8e3d49b17'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:22:17.446383 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-96631267-9adc-4ffd-a9c3-01e8e3d49b17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37b1061b84ec42fda73f6c0779d565d3', 'request_id': 'req-96631267-9adc-4ffd-a9c3-01e8e3d49b17'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:22:17.448960 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:22:17.456643 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-96631267-9adc-4ffd-a9c3-01e8e3d49b17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37b1061b84ec42fda73f6c0779d565d3', 'request_id': 'req-96631267-9adc-4ffd-a9c3-01e8e3d49b17'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:22:17.468296 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-96631267-9adc-4ffd-a9c3-01e8e3d49b17 admin admin] The token cache is being invalidate because project 37b1061b84ec42fda73f6c0779d565d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 09:22:17.468585 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-96631267-9adc-4ffd-a9c3-01e8e3d49b17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:22:17.469370 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-96631267-9adc-4ffd-a9c3-01e8e3d49b17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:22:17.470859 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1205/2424] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 09:22:17 2022] DELETE /identity/v3/projects/37b1061b84ec42fda73f6c0779d565d3 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:22:17.474961 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd75d352-836a-471e-aece-4547b67a4f84 service neutron] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:22:17.485008 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd75d352-836a-471e-aece-4547b67a4f84 service neutron] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:22:17.492536 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:22:17.542554 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e9545a-aa14-48bc-8a63-676659fdc111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:22:17.543180 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e9545a-aa14-48bc-8a63-676659fdc111 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:17.543463 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e9545a-aa14-48bc-8a63-676659fdc111 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:17.543741 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e9545a-aa14-48bc-8a63-676659fdc111 admin admin] PATH_INFO: `/v3/users/4c28652a6b344bf283da93fc8ef45fba` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:17.544845 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e9545a-aa14-48bc-8a63-676659fdc111 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c28652a6b344bf283da93fc8ef45fba)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:22:17.547317 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e9545a-aa14-48bc-8a63-676659fdc111 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:22:17.592929 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-46e9545a-aa14-48bc-8a63-676659fdc111 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c28652a6b344bf283da93fc8ef45fba'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:22:17.593571 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-46e9545a-aa14-48bc-8a63-676659fdc111 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c28652a6b344bf283da93fc8ef45fba'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:22:17.600431 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-46e9545a-aa14-48bc-8a63-676659fdc111 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c28652a6b344bf283da93fc8ef45fba'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:22:17.612998 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-46e9545a-aa14-48bc-8a63-676659fdc111 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c28652a6b344bf283da93fc8ef45fba'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:22:17.620209 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-46e9545a-aa14-48bc-8a63-676659fdc111 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c28652a6b344bf283da93fc8ef45fba'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:22:17.626820 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1220/2425] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 09:22:17 2022] DELETE /identity/v3/users/4c28652a6b344bf283da93fc8ef45fba => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:22:17.892988 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96631267-9adc-4ffd-a9c3-01e8e3d49b17 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:22:17.894453 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-96631267-9adc-4ffd-a9c3-01e8e3d49b17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:22:17.907724 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96631267-9adc-4ffd-a9c3-01e8e3d49b17 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:22:17.965165 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19948185-8cb3-4978-b9d0-81ee17bd31b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:22:17.965839 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19948185-8cb3-4978-b9d0-81ee17bd31b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:17.966091 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19948185-8cb3-4978-b9d0-81ee17bd31b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:17.966366 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19948185-8cb3-4978-b9d0-81ee17bd31b6 admin admin] PATH_INFO: `/v3/projects/2e480e7e000748abb7fa95be606fd0c2` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:17.967473 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19948185-8cb3-4978-b9d0-81ee17bd31b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e480e7e000748abb7fa95be606fd0c2)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:22:17.969903 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19948185-8cb3-4978-b9d0-81ee17bd31b6 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:22:18.014728 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-19948185-8cb3-4978-b9d0-81ee17bd31b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e480e7e000748abb7fa95be606fd0c2', 'request_id': 'req-19948185-8cb3-4978-b9d0-81ee17bd31b6'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:22:18.021032 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-19948185-8cb3-4978-b9d0-81ee17bd31b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e480e7e000748abb7fa95be606fd0c2', 'request_id': 'req-19948185-8cb3-4978-b9d0-81ee17bd31b6'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:22:18.028424 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-19948185-8cb3-4978-b9d0-81ee17bd31b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e480e7e000748abb7fa95be606fd0c2', 'request_id': 'req-19948185-8cb3-4978-b9d0-81ee17bd31b6'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:22:18.039710 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-19948185-8cb3-4978-b9d0-81ee17bd31b6 admin admin] The token cache is being invalidate because project 2e480e7e000748abb7fa95be606fd0c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 09:22:18.040000 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-19948185-8cb3-4978-b9d0-81ee17bd31b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:22:18.040619 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-19948185-8cb3-4978-b9d0-81ee17bd31b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:22:18.042205 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1206/2426] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 09:22:17 2022] DELETE /identity/v3/projects/2e480e7e000748abb7fa95be606fd0c2 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:22:18.046781 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e9545a-aa14-48bc-8a63-676659fdc111 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:22:18.048176 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-46e9545a-aa14-48bc-8a63-676659fdc111 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:22:18.061196 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e9545a-aa14-48bc-8a63-676659fdc111 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:22:18.117523 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d06138-d36b-4cd0-988e-d9eaba94c83c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:22:18.118180 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d06138-d36b-4cd0-988e-d9eaba94c83c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:18.118463 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d06138-d36b-4cd0-988e-d9eaba94c83c admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:18.118740 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d06138-d36b-4cd0-988e-d9eaba94c83c admin admin] PATH_INFO: `/v3/users/f327ca6ca1d34a0caa8f4a5041d1080c` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:18.119838 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d06138-d36b-4cd0-988e-d9eaba94c83c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f327ca6ca1d34a0caa8f4a5041d1080c)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:22:18.121764 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d06138-d36b-4cd0-988e-d9eaba94c83c admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:22:18.153291 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:22:18.167093 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-33d06138-d36b-4cd0-988e-d9eaba94c83c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f327ca6ca1d34a0caa8f4a5041d1080c'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:22:18.167744 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-33d06138-d36b-4cd0-988e-d9eaba94c83c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f327ca6ca1d34a0caa8f4a5041d1080c'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:22:18.174578 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-33d06138-d36b-4cd0-988e-d9eaba94c83c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f327ca6ca1d34a0caa8f4a5041d1080c'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:22:18.186542 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-33d06138-d36b-4cd0-988e-d9eaba94c83c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f327ca6ca1d34a0caa8f4a5041d1080c'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:22:18.193953 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-33d06138-d36b-4cd0-988e-d9eaba94c83c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f327ca6ca1d34a0caa8f4a5041d1080c'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:22:18.199773 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1221/2427] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 09:22:18 2022] DELETE /identity/v3/users/f327ca6ca1d34a0caa8f4a5041d1080c => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:22:18.465590 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19948185-8cb3-4978-b9d0-81ee17bd31b6 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:22:18.466919 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-19948185-8cb3-4978-b9d0-81ee17bd31b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:22:18.480207 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19948185-8cb3-4978-b9d0-81ee17bd31b6 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:22:18.541517 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6738302-c51d-4c1b-bd2e-6f4f604d189d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:22:18.542195 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6738302-c51d-4c1b-bd2e-6f4f604d189d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:18.542478 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6738302-c51d-4c1b-bd2e-6f4f604d189d admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:18.542757 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6738302-c51d-4c1b-bd2e-6f4f604d189d admin admin] PATH_INFO: `/v3/projects/503624d95059472d8dd8172dd393c1f3` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:18.543863 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6738302-c51d-4c1b-bd2e-6f4f604d189d admin admin] RBAC: Authorizing `identity:delete_project(project_id=503624d95059472d8dd8172dd393c1f3)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:22:18.546484 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6738302-c51d-4c1b-bd2e-6f4f604d189d admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:22:18.563229 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:22:18.593241 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-d6738302-c51d-4c1b-bd2e-6f4f604d189d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '503624d95059472d8dd8172dd393c1f3', 'request_id': 'req-d6738302-c51d-4c1b-bd2e-6f4f604d189d'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:22:18.599512 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-d6738302-c51d-4c1b-bd2e-6f4f604d189d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '503624d95059472d8dd8172dd393c1f3', 'request_id': 'req-d6738302-c51d-4c1b-bd2e-6f4f604d189d'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:22:18.607514 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-d6738302-c51d-4c1b-bd2e-6f4f604d189d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '503624d95059472d8dd8172dd393c1f3', 'request_id': 'req-d6738302-c51d-4c1b-bd2e-6f4f604d189d'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:22:18.618835 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-d6738302-c51d-4c1b-bd2e-6f4f604d189d admin admin] The token cache is being invalidate because project 503624d95059472d8dd8172dd393c1f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 09:22:18.619121 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-d6738302-c51d-4c1b-bd2e-6f4f604d189d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:22:18.619737 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-d6738302-c51d-4c1b-bd2e-6f4f604d189d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:22:18.621194 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1207/2428] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 09:22:18 2022] DELETE /identity/v3/projects/503624d95059472d8dd8172dd393c1f3 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:22:18.626228 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8022181-948b-4a77-b5e2-77f903b389c9 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:18.626588 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8022181-948b-4a77-b5e2-77f903b389c9 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:18.626894 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8022181-948b-4a77-b5e2-77f903b389c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:18.646682 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-c8022181-948b-4a77-b5e2-77f903b389c9 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:22:18.647900 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-c8022181-948b-4a77-b5e2-77f903b389c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:22:18.706295 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1222/2429] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:22:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:22:18.713999 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1590ad53-76ec-4cfd-b20c-bd4b7f231ec7 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:18.714284 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1590ad53-76ec-4cfd-b20c-bd4b7f231ec7 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:18.714617 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1590ad53-76ec-4cfd-b20c-bd4b7f231ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:18.735046 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-1590ad53-76ec-4cfd-b20c-bd4b7f231ec7 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:22:18.736234 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-1590ad53-76ec-4cfd-b20c-bd4b7f231ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:22:18.743881 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1208/2430] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:22:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:22:18.747957 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8022181-948b-4a77-b5e2-77f903b389c9 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:22:18.757481 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8022181-948b-4a77-b5e2-77f903b389c9 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:22:18.762262 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba7d1ad-1179-4f8c-86cb-2bda6ba09dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:22:18.762856 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba7d1ad-1179-4f8c-86cb-2bda6ba09dfb admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:18.763136 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba7d1ad-1179-4f8c-86cb-2bda6ba09dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:18.763410 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba7d1ad-1179-4f8c-86cb-2bda6ba09dfb admin admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:18.764050 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba7d1ad-1179-4f8c-86cb-2bda6ba09dfb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 09:22:18.764352 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba7d1ad-1179-4f8c-86cb-2bda6ba09dfb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:22:18.765961 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba7d1ad-1179-4f8c-86cb-2bda6ba09dfb admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:22:18.777512 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1223/2431] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 09:22:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 09:22:18.782101 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1590ad53-76ec-4cfd-b20c-bd4b7f231ec7 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:22:18.790951 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1590ad53-76ec-4cfd-b20c-bd4b7f231ec7 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:22:18.795489 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db18731-8e03-4f80-8091-1f85e558fc2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:22:18.796632 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db18731-8e03-4f80-8091-1f85e558fc2b admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:18.796905 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db18731-8e03-4f80-8091-1f85e558fc2b admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:18.797278 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db18731-8e03-4f80-8091-1f85e558fc2b admin admin] PATH_INFO: `/v3/projects` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:18.797938 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db18731-8e03-4f80-8091-1f85e558fc2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:22:18.800258 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db18731-8e03-4f80-8091-1f85e558fc2b admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:22:18.826290 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1209/2432] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 09:22:18 2022] POST /identity/v3/projects => generated 365 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:22:18.830528 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba7d1ad-1179-4f8c-86cb-2bda6ba09dfb admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:22:18.840656 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba7d1ad-1179-4f8c-86cb-2bda6ba09dfb admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:22:18.895982 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd9ac26-f66e-4bc3-bbe6-e56e78a0ceab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:22:18.896570 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd9ac26-f66e-4bc3-bbe6-e56e78a0ceab admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:18.896849 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd9ac26-f66e-4bc3-bbe6-e56e78a0ceab admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:18.897121 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd9ac26-f66e-4bc3-bbe6-e56e78a0ceab admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:18.897774 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd9ac26-f66e-4bc3-bbe6-e56e78a0ceab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:22:18.899791 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd9ac26-f66e-4bc3-bbe6-e56e78a0ceab admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:22:18.920346 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1224/2433] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 09:22:18 2022] POST /identity/v3/users => generated 283 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:22:18.924431 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db18731-8e03-4f80-8091-1f85e558fc2b admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:22:18.935055 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db18731-8e03-4f80-8091-1f85e558fc2b admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:22:18.939882 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455af67e-228a-4f8c-b2ae-8c6dfa9b3bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:22:18.940483 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455af67e-228a-4f8c-b2ae-8c6dfa9b3bb7 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:18.940780 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455af67e-228a-4f8c-b2ae-8c6dfa9b3bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:18.941053 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455af67e-228a-4f8c-b2ae-8c6dfa9b3bb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:18.941702 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455af67e-228a-4f8c-b2ae-8c6dfa9b3bb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:22:18.943330 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455af67e-228a-4f8c-b2ae-8c6dfa9b3bb7 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:22:18.953661 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1210/2434] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:22:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:22:18.957441 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd9ac26-f66e-4bc3-bbe6-e56e78a0ceab admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:22:18.967199 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd9ac26-f66e-4bc3-bbe6-e56e78a0ceab admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:22:18.972344 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c813efe9-eb8c-4b5a-a1ee-bb052d17a970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:22:18.972929 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c813efe9-eb8c-4b5a-a1ee-bb052d17a970 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:18.973207 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c813efe9-eb8c-4b5a-a1ee-bb052d17a970 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:18.973482 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c813efe9-eb8c-4b5a-a1ee-bb052d17a970 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:18.974153 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c813efe9-eb8c-4b5a-a1ee-bb052d17a970 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:22:18.975726 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c813efe9-eb8c-4b5a-a1ee-bb052d17a970 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:22:18.985550 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1225/2435] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:22:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:22:18.989236 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455af67e-228a-4f8c-b2ae-8c6dfa9b3bb7 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:22:18.999069 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455af67e-228a-4f8c-b2ae-8c6dfa9b3bb7 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:22:19.003695 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6247bd-9250-48e7-a3b6-007db4eac623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:22:19.004322 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6247bd-9250-48e7-a3b6-007db4eac623 admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:19.004604 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6247bd-9250-48e7-a3b6-007db4eac623 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:19.004880 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6247bd-9250-48e7-a3b6-007db4eac623 admin admin] PATH_INFO: `/v3/projects/441ff519239e43a693021f55a7cc14da/users/1afa6eaef1c14cab934e717a0802a111/roles/3248f0268738489b883041851d681bb7` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:19.019748 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6247bd-9250-48e7-a3b6-007db4eac623 admin admin] RBAC: Authorizing `identity:create_grant(project_id=441ff519239e43a693021f55a7cc14da, user_id=1afa6eaef1c14cab934e717a0802a111, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:22:19.022418 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6247bd-9250-48e7-a3b6-007db4eac623 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:22:19.035566 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1211/2436] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:22:18 2022] PUT /identity/v3/projects/441ff519239e43a693021f55a7cc14da/users/1afa6eaef1c14cab934e717a0802a111/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:22:19.046675 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f898bf-5242-49eb-8eb8-325eb08b430b None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:19.046966 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f898bf-5242-49eb-8eb8-325eb08b430b None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:19.047541 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f898bf-5242-49eb-8eb8-325eb08b430b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:19.075573 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-b6f898bf-5242-49eb-8eb8-325eb08b430b None None] MFA Rules not processed for user `1afa6eaef1c14cab934e717a0802a111`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:22:19.076776 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-b6f898bf-5242-49eb-8eb8-325eb08b430b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:22:19.081235 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:22:19.142311 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1226/2437] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:22:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:22:19.153473 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6247bd-9250-48e7-a3b6-007db4eac623 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:22:19.155778 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-7d6247bd-9250-48e7-a3b6-007db4eac623 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:22:19.210758 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6247bd-9250-48e7-a3b6-007db4eac623 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:22:19.214341 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fae2f56-c45b-4fbc-8eb6-1b517f339e80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:22:19.214942 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae2f56-c45b-4fbc-8eb6-1b517f339e80 service cinder] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:19.215221 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae2f56-c45b-4fbc-8eb6-1b517f339e80 service cinder] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:19.215496 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae2f56-c45b-4fbc-8eb6-1b517f339e80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:19.224177 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fae2f56-c45b-4fbc-8eb6-1b517f339e80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:22:19.225586 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fae2f56-c45b-4fbc-8eb6-1b517f339e80 service cinder] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:22:19.234273 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1212/2438] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:22:19 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:22:29.138831 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=25 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:22:29.230202 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=70,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:22:53.150063 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f898bf-5242-49eb-8eb8-325eb08b430b None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:22:53.210682 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f898bf-5242-49eb-8eb8-325eb08b430b None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:22:53.215125 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d8a74f-80a7-4b1c-b956-c6eb6fef0cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:22:53.215749 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d8a74f-80a7-4b1c-b956-c6eb6fef0cc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:53.216031 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d8a74f-80a7-4b1c-b956-c6eb6fef0cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:53.216311 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d8a74f-80a7-4b1c-b956-c6eb6fef0cc6 admin admin] PATH_INFO: `/v3/users/1afa6eaef1c14cab934e717a0802a111` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:53.217407 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d8a74f-80a7-4b1c-b956-c6eb6fef0cc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1afa6eaef1c14cab934e717a0802a111)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:22:53.219352 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d8a74f-80a7-4b1c-b956-c6eb6fef0cc6 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:22:53.264483 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d6d8a74f-80a7-4b1c-b956-c6eb6fef0cc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1afa6eaef1c14cab934e717a0802a111'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:22:53.265124 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d6d8a74f-80a7-4b1c-b956-c6eb6fef0cc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1afa6eaef1c14cab934e717a0802a111'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:22:53.272734 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d6d8a74f-80a7-4b1c-b956-c6eb6fef0cc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1afa6eaef1c14cab934e717a0802a111'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:22:53.284702 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d6d8a74f-80a7-4b1c-b956-c6eb6fef0cc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1afa6eaef1c14cab934e717a0802a111'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:22:53.291697 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d6d8a74f-80a7-4b1c-b956-c6eb6fef0cc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1afa6eaef1c14cab934e717a0802a111'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:22:53.297509 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1227/2439] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 09:22:53 2022] DELETE /identity/v3/users/1afa6eaef1c14cab934e717a0802a111 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:22:53.302745 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7da02bc-64d9-49dc-9740-5cac99d259bf None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:53.303221 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7da02bc-64d9-49dc-9740-5cac99d259bf None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:53.303332 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7da02bc-64d9-49dc-9740-5cac99d259bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:53.322980 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-d7da02bc-64d9-49dc-9740-5cac99d259bf None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:22:53.324268 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-d7da02bc-64d9-49dc-9740-5cac99d259bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:22:53.381877 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1213/2440] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:22:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:22:53.391790 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d8a74f-80a7-4b1c-b956-c6eb6fef0cc6 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:22:53.393207 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-d6d8a74f-80a7-4b1c-b956-c6eb6fef0cc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:22:53.450757 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d8a74f-80a7-4b1c-b956-c6eb6fef0cc6 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:22:53.454389 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a695b7-82c3-4bef-97fb-fe4b7dd8ea28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:22:53.455036 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a695b7-82c3-4bef-97fb-fe4b7dd8ea28 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:53.455318 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a695b7-82c3-4bef-97fb-fe4b7dd8ea28 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:53.455593 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a695b7-82c3-4bef-97fb-fe4b7dd8ea28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:53.464365 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a695b7-82c3-4bef-97fb-fe4b7dd8ea28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:22:53.465794 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a695b7-82c3-4bef-97fb-fe4b7dd8ea28 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:22:53.475112 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1228/2441] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:22:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:22:53.734318 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7da02bc-64d9-49dc-9740-5cac99d259bf None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:22:53.735752 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-d7da02bc-64d9-49dc-9740-5cac99d259bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:22:53.748378 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7da02bc-64d9-49dc-9740-5cac99d259bf None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:22:53.752959 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc119d9d-d9ab-4595-9941-2afcdca2ad4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:22:53.753584 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc119d9d-d9ab-4595-9941-2afcdca2ad4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:53.754105 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc119d9d-d9ab-4595-9941-2afcdca2ad4b admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:53.754388 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc119d9d-d9ab-4595-9941-2afcdca2ad4b admin admin] PATH_INFO: `/v3/projects/441ff519239e43a693021f55a7cc14da` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:53.755504 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc119d9d-d9ab-4595-9941-2afcdca2ad4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=441ff519239e43a693021f55a7cc14da)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:22:53.757410 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc119d9d-d9ab-4595-9941-2afcdca2ad4b admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:22:53.804594 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-fc119d9d-d9ab-4595-9941-2afcdca2ad4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '441ff519239e43a693021f55a7cc14da', 'request_id': 'req-fc119d9d-d9ab-4595-9941-2afcdca2ad4b'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:22:53.810710 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-fc119d9d-d9ab-4595-9941-2afcdca2ad4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '441ff519239e43a693021f55a7cc14da', 'request_id': 'req-fc119d9d-d9ab-4595-9941-2afcdca2ad4b'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:22:53.817642 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-fc119d9d-d9ab-4595-9941-2afcdca2ad4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '441ff519239e43a693021f55a7cc14da', 'request_id': 'req-fc119d9d-d9ab-4595-9941-2afcdca2ad4b'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:22:53.827738 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-fc119d9d-d9ab-4595-9941-2afcdca2ad4b admin admin] The token cache is being invalidate because project 441ff519239e43a693021f55a7cc14da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 09:22:53.828023 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-fc119d9d-d9ab-4595-9941-2afcdca2ad4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:22:53.828635 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-fc119d9d-d9ab-4595-9941-2afcdca2ad4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:22:53.830140 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1214/2442] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 09:22:53 2022] DELETE /identity/v3/projects/441ff519239e43a693021f55a7cc14da => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:22:53.835064 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10072600-3fb7-4c06-b94d-5a6f91b9437e None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:53.835356 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10072600-3fb7-4c06-b94d-5a6f91b9437e None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:53.835640 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10072600-3fb7-4c06-b94d-5a6f91b9437e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:53.855327 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-10072600-3fb7-4c06-b94d-5a6f91b9437e None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:22:53.856546 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-10072600-3fb7-4c06-b94d-5a6f91b9437e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:22:53.874599 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:22:53.918370 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1229/2443] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:22:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:22:53.926933 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07a8fe-5722-4e70-94b8-421bd74be3ef None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:53.927247 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07a8fe-5722-4e70-94b8-421bd74be3ef None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:53.927524 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07a8fe-5722-4e70-94b8-421bd74be3ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:53.946938 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-ff07a8fe-5722-4e70-94b8-421bd74be3ef None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:22:53.948119 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-ff07a8fe-5722-4e70-94b8-421bd74be3ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:22:53.955775 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1215/2444] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:22:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:22:53.959971 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10072600-3fb7-4c06-b94d-5a6f91b9437e None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:22:53.969519 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10072600-3fb7-4c06-b94d-5a6f91b9437e None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:22:53.974635 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b53478-65c5-4fa3-b666-38d548aef79a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:22:53.975329 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b53478-65c5-4fa3-b666-38d548aef79a admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:53.975620 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b53478-65c5-4fa3-b666-38d548aef79a admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:53.975899 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b53478-65c5-4fa3-b666-38d548aef79a admin admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:53.976547 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b53478-65c5-4fa3-b666-38d548aef79a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 09:22:53.976857 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b53478-65c5-4fa3-b666-38d548aef79a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:22:53.978482 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b53478-65c5-4fa3-b666-38d548aef79a admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:22:53.990883 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1230/2445] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 09:22:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 09:22:53.994402 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff07a8fe-5722-4e70-94b8-421bd74be3ef None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:22:54.003569 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff07a8fe-5722-4e70-94b8-421bd74be3ef None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:22:54.008074 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7510152-c059-498b-9971-5e4d37eb2199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:22:54.008719 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7510152-c059-498b-9971-5e4d37eb2199 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:54.008998 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7510152-c059-498b-9971-5e4d37eb2199 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:54.009273 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7510152-c059-498b-9971-5e4d37eb2199 admin admin] PATH_INFO: `/v3/projects` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:54.009931 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7510152-c059-498b-9971-5e4d37eb2199 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:22:54.011857 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7510152-c059-498b-9971-5e4d37eb2199 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:22:54.039319 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1216/2446] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 09:22:53 2022] POST /identity/v3/projects => generated 391 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:22:54.043479 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b53478-65c5-4fa3-b666-38d548aef79a admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:22:54.054488 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b53478-65c5-4fa3-b666-38d548aef79a admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:22:54.110690 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8ef752-6ed9-4d11-83ff-ddd3f98be2ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:22:54.111351 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ef752-6ed9-4d11-83ff-ddd3f98be2ba admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:54.111635 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ef752-6ed9-4d11-83ff-ddd3f98be2ba admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:54.111913 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ef752-6ed9-4d11-83ff-ddd3f98be2ba admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:54.112555 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8ef752-6ed9-4d11-83ff-ddd3f98be2ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:22:54.114502 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8ef752-6ed9-4d11-83ff-ddd3f98be2ba admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:22:54.134864 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1231/2447] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 09:22:54 2022] POST /identity/v3/users => generated 296 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 09:22:54.138877 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7510152-c059-498b-9971-5e4d37eb2199 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:22:54.149445 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7510152-c059-498b-9971-5e4d37eb2199 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:22:54.153929 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dacc23-aaa5-4ff3-b6b4-079a44ddde73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:22:54.154517 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dacc23-aaa5-4ff3-b6b4-079a44ddde73 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:54.154789 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dacc23-aaa5-4ff3-b6b4-079a44ddde73 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:54.155058 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dacc23-aaa5-4ff3-b6b4-079a44ddde73 admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:54.155683 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dacc23-aaa5-4ff3-b6b4-079a44ddde73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:22:54.157329 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dacc23-aaa5-4ff3-b6b4-079a44ddde73 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:22:54.167523 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1217/2448] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:22:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 09:22:54.171518 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8ef752-6ed9-4d11-83ff-ddd3f98be2ba admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:22:54.181937 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8ef752-6ed9-4d11-83ff-ddd3f98be2ba admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:22:54.186708 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90220a7e-0142-4183-8a8e-69c91e69657c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:22:54.187302 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90220a7e-0142-4183-8a8e-69c91e69657c admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:54.187590 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90220a7e-0142-4183-8a8e-69c91e69657c admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:54.187865 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90220a7e-0142-4183-8a8e-69c91e69657c admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:54.188529 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90220a7e-0142-4183-8a8e-69c91e69657c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:22:54.190235 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90220a7e-0142-4183-8a8e-69c91e69657c admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:22:54.200981 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1232/2449] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:22:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 09:22:54.206033 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dacc23-aaa5-4ff3-b6b4-079a44ddde73 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:22:54.216090 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dacc23-aaa5-4ff3-b6b4-079a44ddde73 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:22:54.221264 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1158d3-dfcb-4d11-8df1-00bcc4d6adea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:22:54.221950 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1158d3-dfcb-4d11-8df1-00bcc4d6adea admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:54.222228 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1158d3-dfcb-4d11-8df1-00bcc4d6adea admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:54.222524 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1158d3-dfcb-4d11-8df1-00bcc4d6adea admin admin] PATH_INFO: `/v3/projects/a20e86d5fde74aaaac624cf065372183/users/d2af83f18f244bc9a88097865df43fdb/roles/3248f0268738489b883041851d681bb7` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:54.237898 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1158d3-dfcb-4d11-8df1-00bcc4d6adea admin admin] RBAC: Authorizing `identity:create_grant(project_id=a20e86d5fde74aaaac624cf065372183, user_id=d2af83f18f244bc9a88097865df43fdb, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:22:54.240707 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1158d3-dfcb-4d11-8df1-00bcc4d6adea admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:22:54.254493 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1218/2450] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:22:54 2022] PUT /identity/v3/projects/a20e86d5fde74aaaac624cf065372183/users/d2af83f18f244bc9a88097865df43fdb/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:22:54.265993 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194150dc-7e59-426a-a4e6-520c4dacaa9a None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:54.266320 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194150dc-7e59-426a-a4e6-520c4dacaa9a None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:54.266599 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194150dc-7e59-426a-a4e6-520c4dacaa9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:54.293995 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-194150dc-7e59-426a-a4e6-520c4dacaa9a None None] MFA Rules not processed for user `d2af83f18f244bc9a88097865df43fdb`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:22:54.295216 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-194150dc-7e59-426a-a4e6-520c4dacaa9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:22:54.360082 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1233/2451] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:22:54 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 09:22:54.371362 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1158d3-dfcb-4d11-8df1-00bcc4d6adea admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:22:54.373130 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-ed1158d3-dfcb-4d11-8df1-00bcc4d6adea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:22:54.398013 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:22:54.436232 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1158d3-dfcb-4d11-8df1-00bcc4d6adea admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:22:54.441625 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5686785-efac-432b-8306-5b14c1dd7850 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:22:54.441891 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5686785-efac-432b-8306-5b14c1dd7850 service cinder] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:54.442171 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5686785-efac-432b-8306-5b14c1dd7850 service cinder] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:54.442443 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5686785-efac-432b-8306-5b14c1dd7850 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:54.451274 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5686785-efac-432b-8306-5b14c1dd7850 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:22:54.452695 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5686785-efac-432b-8306-5b14c1dd7850 service cinder] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:22:54.461899 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1219/2452] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:22:54 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:22:55.257032 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194150dc-7e59-426a-a4e6-520c4dacaa9a None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:22:55.258695 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-194150dc-7e59-426a-a4e6-520c4dacaa9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:22:55.285476 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:22:55.349452 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194150dc-7e59-426a-a4e6-520c4dacaa9a None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:22:55.355155 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5be83f1-a408-43ae-8117-c1a2ef86d56d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0a8af5cadc4cc1ab73e8eb2a20bb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:22:55.355786 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be83f1-a408-43ae-8117-c1a2ef86d56d service nova] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:22:55.356071 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be83f1-a408-43ae-8117-c1a2ef86d56d service nova] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:22:55.356348 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be83f1-a408-43ae-8117-c1a2ef86d56d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:22:55.358159 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-a5be83f1-a408-43ae-8117-c1a2ef86d56d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:22:55.372921 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5be83f1-a408-43ae-8117-c1a2ef86d56d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:22:55.374888 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5be83f1-a408-43ae-8117-c1a2ef86d56d service nova] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:22:55.403254 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1234/2453] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:22:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:23:04.457608 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=17 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:23:05.399178 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=26 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:23:10.327315 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5686785-efac-432b-8306-5b14c1dd7850 service cinder] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:23:10.329032 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-f5686785-efac-432b-8306-5b14c1dd7850 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:23:10.411798 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5686785-efac-432b-8306-5b14c1dd7850 service cinder] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:23:10.416237 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da46eb29-7136-4960-823d-d018f5d0d9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:23:10.416889 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da46eb29-7136-4960-823d-d018f5d0d9ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:23:10.417168 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da46eb29-7136-4960-823d-d018f5d0d9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:23:10.417439 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da46eb29-7136-4960-823d-d018f5d0d9ae admin admin] PATH_INFO: `/v3/users/440ce33b27fa46f6aaeab9184d8ef5a3` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:23:10.418587 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da46eb29-7136-4960-823d-d018f5d0d9ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=440ce33b27fa46f6aaeab9184d8ef5a3)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:23:10.420498 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da46eb29-7136-4960-823d-d018f5d0d9ae admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:23:10.468298 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-da46eb29-7136-4960-823d-d018f5d0d9ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '440ce33b27fa46f6aaeab9184d8ef5a3'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:23:10.468910 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-da46eb29-7136-4960-823d-d018f5d0d9ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '440ce33b27fa46f6aaeab9184d8ef5a3'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:23:10.482392 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-da46eb29-7136-4960-823d-d018f5d0d9ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '440ce33b27fa46f6aaeab9184d8ef5a3'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:23:10.491710 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-da46eb29-7136-4960-823d-d018f5d0d9ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '440ce33b27fa46f6aaeab9184d8ef5a3'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:23:10.492387 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-da46eb29-7136-4960-823d-d018f5d0d9ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '440ce33b27fa46f6aaeab9184d8ef5a3'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:23:10.505088 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1220/2454] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 09:23:10 2022] DELETE /identity/v3/users/440ce33b27fa46f6aaeab9184d8ef5a3 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:23:10.692786 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5be83f1-a408-43ae-8117-c1a2ef86d56d service nova] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:23:10.710162 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5be83f1-a408-43ae-8117-c1a2ef86d56d service nova] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:23:10.773802 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aefe86-cbf9-4201-8108-ce575d7c3075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:23:10.774415 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aefe86-cbf9-4201-8108-ce575d7c3075 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:23:10.774696 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aefe86-cbf9-4201-8108-ce575d7c3075 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:23:10.774977 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aefe86-cbf9-4201-8108-ce575d7c3075 admin admin] PATH_INFO: `/v3/projects/6c1e74e55af54016b9e63666c1594675` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:23:10.776082 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05aefe86-cbf9-4201-8108-ce575d7c3075 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c1e74e55af54016b9e63666c1594675)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:23:10.778046 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05aefe86-cbf9-4201-8108-ce575d7c3075 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:23:10.826167 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-05aefe86-cbf9-4201-8108-ce575d7c3075 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c1e74e55af54016b9e63666c1594675', 'request_id': 'req-05aefe86-cbf9-4201-8108-ce575d7c3075'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:23:10.833066 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-05aefe86-cbf9-4201-8108-ce575d7c3075 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c1e74e55af54016b9e63666c1594675', 'request_id': 'req-05aefe86-cbf9-4201-8108-ce575d7c3075'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:23:10.845334 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-05aefe86-cbf9-4201-8108-ce575d7c3075 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c1e74e55af54016b9e63666c1594675', 'request_id': 'req-05aefe86-cbf9-4201-8108-ce575d7c3075'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:23:10.857024 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-05aefe86-cbf9-4201-8108-ce575d7c3075 admin admin] The token cache is being invalidate because project 6c1e74e55af54016b9e63666c1594675 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 09:23:10.857354 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-05aefe86-cbf9-4201-8108-ce575d7c3075 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:23:10.858151 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-05aefe86-cbf9-4201-8108-ce575d7c3075 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:23:10.859931 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1235/2455] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 09:23:10 2022] DELETE /identity/v3/projects/6c1e74e55af54016b9e63666c1594675 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:23:10.864081 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da46eb29-7136-4960-823d-d018f5d0d9ae admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:23:10.865495 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-da46eb29-7136-4960-823d-d018f5d0d9ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:23:10.878860 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da46eb29-7136-4960-823d-d018f5d0d9ae admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:23:10.933020 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fc23ed-751c-4a7f-b0b3-2eaca96a8de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:23:10.933637 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc23ed-751c-4a7f-b0b3-2eaca96a8de6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:23:10.933949 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc23ed-751c-4a7f-b0b3-2eaca96a8de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:23:10.934219 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc23ed-751c-4a7f-b0b3-2eaca96a8de6 admin admin] PATH_INFO: `/v3/users/b37af484a4464540823d1eedcb14ada9` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:23:10.935322 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fc23ed-751c-4a7f-b0b3-2eaca96a8de6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b37af484a4464540823d1eedcb14ada9)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:23:10.937255 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fc23ed-751c-4a7f-b0b3-2eaca96a8de6 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:23:10.958974 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:23:10.984288 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-98fc23ed-751c-4a7f-b0b3-2eaca96a8de6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b37af484a4464540823d1eedcb14ada9'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:23:10.984941 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-98fc23ed-751c-4a7f-b0b3-2eaca96a8de6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b37af484a4464540823d1eedcb14ada9'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:23:10.997132 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-98fc23ed-751c-4a7f-b0b3-2eaca96a8de6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b37af484a4464540823d1eedcb14ada9'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:23:11.004702 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-98fc23ed-751c-4a7f-b0b3-2eaca96a8de6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b37af484a4464540823d1eedcb14ada9'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:23:11.005476 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-98fc23ed-751c-4a7f-b0b3-2eaca96a8de6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b37af484a4464540823d1eedcb14ada9'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:23:11.017643 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1221/2456] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 09:23:10 2022] DELETE /identity/v3/users/b37af484a4464540823d1eedcb14ada9 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:23:11.185947 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aefe86-cbf9-4201-8108-ce575d7c3075 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:23:11.187499 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-05aefe86-cbf9-4201-8108-ce575d7c3075 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:23:11.200985 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aefe86-cbf9-4201-8108-ce575d7c3075 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:23:11.257401 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02a1e4a-fcb6-4ead-a073-e0b23956bb11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:23:11.258166 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a1e4a-fcb6-4ead-a073-e0b23956bb11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:23:11.258457 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a1e4a-fcb6-4ead-a073-e0b23956bb11 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:23:11.258738 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a1e4a-fcb6-4ead-a073-e0b23956bb11 admin admin] PATH_INFO: `/v3/projects/e05b8802303a4d3d8c2fe66e6af0e604` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:23:11.259879 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02a1e4a-fcb6-4ead-a073-e0b23956bb11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e05b8802303a4d3d8c2fe66e6af0e604)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:23:11.261842 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02a1e4a-fcb6-4ead-a073-e0b23956bb11 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:23:11.296070 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:23:11.310842 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-e02a1e4a-fcb6-4ead-a073-e0b23956bb11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e05b8802303a4d3d8c2fe66e6af0e604', 'request_id': 'req-e02a1e4a-fcb6-4ead-a073-e0b23956bb11'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:23:11.317594 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-e02a1e4a-fcb6-4ead-a073-e0b23956bb11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e05b8802303a4d3d8c2fe66e6af0e604', 'request_id': 'req-e02a1e4a-fcb6-4ead-a073-e0b23956bb11'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:23:11.324649 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-e02a1e4a-fcb6-4ead-a073-e0b23956bb11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e05b8802303a4d3d8c2fe66e6af0e604', 'request_id': 'req-e02a1e4a-fcb6-4ead-a073-e0b23956bb11'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:23:11.334794 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-e02a1e4a-fcb6-4ead-a073-e0b23956bb11 admin admin] The token cache is being invalidate because project e05b8802303a4d3d8c2fe66e6af0e604 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 09:23:11.335076 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-e02a1e4a-fcb6-4ead-a073-e0b23956bb11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:23:11.335693 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-e02a1e4a-fcb6-4ead-a073-e0b23956bb11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:23:11.337145 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1236/2457] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 09:23:11 2022] DELETE /identity/v3/projects/e05b8802303a4d3d8c2fe66e6af0e604 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:23:12.886227 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fc23ed-751c-4a7f-b0b3-2eaca96a8de6 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:23:12.887618 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-98fc23ed-751c-4a7f-b0b3-2eaca96a8de6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:23:12.900109 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fc23ed-751c-4a7f-b0b3-2eaca96a8de6 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:23:12.954682 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063b654e-3b80-475f-8120-110b266c09a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:23:12.955235 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b654e-3b80-475f-8120-110b266c09a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:23:12.955550 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b654e-3b80-475f-8120-110b266c09a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:23:12.955842 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b654e-3b80-475f-8120-110b266c09a7 admin admin] PATH_INFO: `/v3/users/d2af83f18f244bc9a88097865df43fdb` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:23:12.956962 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063b654e-3b80-475f-8120-110b266c09a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2af83f18f244bc9a88097865df43fdb)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:23:12.958892 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063b654e-3b80-475f-8120-110b266c09a7 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:23:13.003424 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-063b654e-3b80-475f-8120-110b266c09a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2af83f18f244bc9a88097865df43fdb'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:23:13.004054 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-063b654e-3b80-475f-8120-110b266c09a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2af83f18f244bc9a88097865df43fdb'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:23:13.015495 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-063b654e-3b80-475f-8120-110b266c09a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2af83f18f244bc9a88097865df43fdb'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:23:13.022214 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-063b654e-3b80-475f-8120-110b266c09a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2af83f18f244bc9a88097865df43fdb'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:23:13.022819 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-063b654e-3b80-475f-8120-110b266c09a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2af83f18f244bc9a88097865df43fdb'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:23:13.034527 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1222/2458] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 09:23:12 2022] DELETE /identity/v3/users/d2af83f18f244bc9a88097865df43fdb => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:23:13.039040 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10289c45-f890-4516-8609-346012b79fe8 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:23:13.039549 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10289c45-f890-4516-8609-346012b79fe8 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:23:13.039654 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10289c45-f890-4516-8609-346012b79fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:23:13.059184 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-10289c45-f890-4516-8609-346012b79fe8 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:23:13.060394 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-10289c45-f890-4516-8609-346012b79fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:23:13.118800 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1237/2459] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:23:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:23:13.129205 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063b654e-3b80-475f-8120-110b266c09a7 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:23:13.130602 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-063b654e-3b80-475f-8120-110b266c09a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:23:13.150156 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:23:13.189274 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063b654e-3b80-475f-8120-110b266c09a7 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:23:13.192884 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b215034a-a3e9-4bae-b74d-f56b0b5167bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:23:13.193483 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b215034a-a3e9-4bae-b74d-f56b0b5167bb service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:23:13.193790 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b215034a-a3e9-4bae-b74d-f56b0b5167bb service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:23:13.194068 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b215034a-a3e9-4bae-b74d-f56b0b5167bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:23:13.202872 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b215034a-a3e9-4bae-b74d-f56b0b5167bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:23:13.204297 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b215034a-a3e9-4bae-b74d-f56b0b5167bb service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:23:13.213633 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1223/2460] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:23:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:23:13.477741 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10289c45-f890-4516-8609-346012b79fe8 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:23:13.479239 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-10289c45-f890-4516-8609-346012b79fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:23:13.491941 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10289c45-f890-4516-8609-346012b79fe8 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:23:13.497134 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5381185-b47e-40e8-8f3b-0ed5ad9c2545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:23:13.497878 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5381185-b47e-40e8-8f3b-0ed5ad9c2545 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:23:13.498167 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5381185-b47e-40e8-8f3b-0ed5ad9c2545 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:23:13.498441 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5381185-b47e-40e8-8f3b-0ed5ad9c2545 admin admin] PATH_INFO: `/v3/projects/a20e86d5fde74aaaac624cf065372183` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:23:13.499576 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5381185-b47e-40e8-8f3b-0ed5ad9c2545 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a20e86d5fde74aaaac624cf065372183)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:23:13.501465 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5381185-b47e-40e8-8f3b-0ed5ad9c2545 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:23:13.548437 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d5381185-b47e-40e8-8f3b-0ed5ad9c2545 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a20e86d5fde74aaaac624cf065372183', 'request_id': 'req-d5381185-b47e-40e8-8f3b-0ed5ad9c2545'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:23:13.555001 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d5381185-b47e-40e8-8f3b-0ed5ad9c2545 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a20e86d5fde74aaaac624cf065372183', 'request_id': 'req-d5381185-b47e-40e8-8f3b-0ed5ad9c2545'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:23:13.561761 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d5381185-b47e-40e8-8f3b-0ed5ad9c2545 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a20e86d5fde74aaaac624cf065372183', 'request_id': 'req-d5381185-b47e-40e8-8f3b-0ed5ad9c2545'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:23:13.572060 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d5381185-b47e-40e8-8f3b-0ed5ad9c2545 admin admin] The token cache is being invalidate because project a20e86d5fde74aaaac624cf065372183 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 09:23:13.572347 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d5381185-b47e-40e8-8f3b-0ed5ad9c2545 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:23:13.572964 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d5381185-b47e-40e8-8f3b-0ed5ad9c2545 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:23:13.574465 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1238/2461] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 09:23:13 2022] DELETE /identity/v3/projects/a20e86d5fde74aaaac624cf065372183 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:23:13.579551 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c6cf4-10fe-41f5-8fa0-3f1029e0b626 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:23:13.579866 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c6cf4-10fe-41f5-8fa0-3f1029e0b626 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:23:13.580146 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c6cf4-10fe-41f5-8fa0-3f1029e0b626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:23:13.600658 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-cd0c6cf4-10fe-41f5-8fa0-3f1029e0b626 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:23:13.601894 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-cd0c6cf4-10fe-41f5-8fa0-3f1029e0b626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:23:13.659804 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1224/2462] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:23:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:23:13.667607 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234fa4fa-5280-4054-b370-e3a040435ea3 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:23:13.667913 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234fa4fa-5280-4054-b370-e3a040435ea3 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:23:13.668200 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234fa4fa-5280-4054-b370-e3a040435ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:23:13.687628 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-234fa4fa-5280-4054-b370-e3a040435ea3 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:23:13.688813 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-234fa4fa-5280-4054-b370-e3a040435ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:23:13.696808 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1239/2463] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:23:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 09:23:13.700839 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0c6cf4-10fe-41f5-8fa0-3f1029e0b626 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:23:13.711018 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0c6cf4-10fe-41f5-8fa0-3f1029e0b626 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:23:13.716006 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9dc178-55b4-4f48-8d55-2741dfced5e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:23:13.716609 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9dc178-55b4-4f48-8d55-2741dfced5e3 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:23:13.716894 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9dc178-55b4-4f48-8d55-2741dfced5e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:23:13.717173 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9dc178-55b4-4f48-8d55-2741dfced5e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:23:13.717870 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9dc178-55b4-4f48-8d55-2741dfced5e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 09:23:13.718185 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9dc178-55b4-4f48-8d55-2741dfced5e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:23:13.719849 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9dc178-55b4-4f48-8d55-2741dfced5e3 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:23:13.735151 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1225/2464] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 09:23:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 09:23:13.736199 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234fa4fa-5280-4054-b370-e3a040435ea3 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:23:13.745375 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234fa4fa-5280-4054-b370-e3a040435ea3 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:23:13.750200 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2742787-8a0d-4408-8713-715462476566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:23:13.750809 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2742787-8a0d-4408-8713-715462476566 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:23:13.751094 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2742787-8a0d-4408-8713-715462476566 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:23:13.751373 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2742787-8a0d-4408-8713-715462476566 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:23:13.752025 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2742787-8a0d-4408-8713-715462476566 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:23:13.758965 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2742787-8a0d-4408-8713-715462476566 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:23:13.784676 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1240/2465] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 09:23:13 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 09:23:13.788796 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9dc178-55b4-4f48-8d55-2741dfced5e3 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:23:13.798801 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9dc178-55b4-4f48-8d55-2741dfced5e3 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:23:13.855445 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c828434-754f-47a3-b1e5-e9327f072bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:23:13.856042 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c828434-754f-47a3-b1e5-e9327f072bf5 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:23:13.856323 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c828434-754f-47a3-b1e5-e9327f072bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:23:13.856600 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c828434-754f-47a3-b1e5-e9327f072bf5 admin admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:23:13.857214 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c828434-754f-47a3-b1e5-e9327f072bf5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:23:13.859164 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c828434-754f-47a3-b1e5-e9327f072bf5 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:23:13.880463 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1226/2466] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 09:23:13 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:23:13.884579 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2742787-8a0d-4408-8713-715462476566 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:23:13.895439 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2742787-8a0d-4408-8713-715462476566 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:23:13.899954 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0581cbaf-f24d-4baf-af64-8f9a81a0fa44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:23:13.900545 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0581cbaf-f24d-4baf-af64-8f9a81a0fa44 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:23:13.900824 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0581cbaf-f24d-4baf-af64-8f9a81a0fa44 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:23:13.901097 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0581cbaf-f24d-4baf-af64-8f9a81a0fa44 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:23:13.901761 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0581cbaf-f24d-4baf-af64-8f9a81a0fa44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:23:13.903339 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0581cbaf-f24d-4baf-af64-8f9a81a0fa44 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:23:13.913301 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1241/2467] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:23:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:23:13.917017 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c828434-754f-47a3-b1e5-e9327f072bf5 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:23:13.927188 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c828434-754f-47a3-b1e5-e9327f072bf5 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:23:13.931720 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7247b04-a5d6-412e-a568-8aa383555bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:23:13.932308 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7247b04-a5d6-412e-a568-8aa383555bb2 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:23:13.932591 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7247b04-a5d6-412e-a568-8aa383555bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:23:13.932867 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7247b04-a5d6-412e-a568-8aa383555bb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:23:13.933499 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7247b04-a5d6-412e-a568-8aa383555bb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:23:13.935137 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7247b04-a5d6-412e-a568-8aa383555bb2 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:23:13.946065 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1227/2468] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:23:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:23:13.949863 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0581cbaf-f24d-4baf-af64-8f9a81a0fa44 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:23:13.956749 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:23:13.960573 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0581cbaf-f24d-4baf-af64-8f9a81a0fa44 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:23:13.966445 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e666611b-7a28-4a6d-ba60-13f7dded8f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:23:13.967249 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e666611b-7a28-4a6d-ba60-13f7dded8f9a admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:23:13.967738 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e666611b-7a28-4a6d-ba60-13f7dded8f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:23:13.968072 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e666611b-7a28-4a6d-ba60-13f7dded8f9a admin admin] PATH_INFO: `/v3/projects/c6262387226346b7a736e34171afb4de/users/04b8dc0532de441db59bd4fe465a8c0a/roles/3248f0268738489b883041851d681bb7` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:23:13.982428 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e666611b-7a28-4a6d-ba60-13f7dded8f9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6262387226346b7a736e34171afb4de, user_id=04b8dc0532de441db59bd4fe465a8c0a, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:23:13.984980 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e666611b-7a28-4a6d-ba60-13f7dded8f9a admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:23:13.998078 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1242/2469] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:23:13 2022] PUT /identity/v3/projects/c6262387226346b7a736e34171afb4de/users/04b8dc0532de441db59bd4fe465a8c0a/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:23:14.003570 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef467fc0-0034-4646-a221-63ee7aaa08b2 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:23:14.003859 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef467fc0-0034-4646-a221-63ee7aaa08b2 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:23:14.004137 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef467fc0-0034-4646-a221-63ee7aaa08b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:23:14.023124 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-ef467fc0-0034-4646-a221-63ee7aaa08b2 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:23:14.024319 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-ef467fc0-0034-4646-a221-63ee7aaa08b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:23:14.043777 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:23:14.083819 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1228/2470] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:23:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:23:14.093366 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e666611b-7a28-4a6d-ba60-13f7dded8f9a admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:23:14.095141 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-e666611b-7a28-4a6d-ba60-13f7dded8f9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:23:14.150528 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e666611b-7a28-4a6d-ba60-13f7dded8f9a admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:23:14.154120 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b69215-aaad-4df5-9aed-19a4af3dee10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:23:14.154717 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b69215-aaad-4df5-9aed-19a4af3dee10 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:23:14.154997 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b69215-aaad-4df5-9aed-19a4af3dee10 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:23:14.155270 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b69215-aaad-4df5-9aed-19a4af3dee10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:23:14.164489 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b69215-aaad-4df5-9aed-19a4af3dee10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:23:14.165932 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b69215-aaad-4df5-9aed-19a4af3dee10 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:23:14.175188 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1243/2471] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:23:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:23:17.856759 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d38bb1-2e2b-44d8-bd72-0ad7d4ef9952 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:23:17.857303 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d38bb1-2e2b-44d8-bd72-0ad7d4ef9952 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:23:17.857303 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d38bb1-2e2b-44d8-bd72-0ad7d4ef9952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:23:17.894117 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-68d38bb1-2e2b-44d8-bd72-0ad7d4ef9952 None None] MFA Rules not processed for user `04b8dc0532de441db59bd4fe465a8c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:23:17.895726 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-68d38bb1-2e2b-44d8-bd72-0ad7d4ef9952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:23:17.967703 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1229/2472] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:23:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:23:17.968719 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b69215-aaad-4df5-9aed-19a4af3dee10 service neutron] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:23:17.982556 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b69215-aaad-4df5-9aed-19a4af3dee10 service neutron] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:23:17.988248 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056cfdb5-5a9d-4cbe-927e-36cc81e844de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:23:17.988993 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056cfdb5-5a9d-4cbe-927e-36cc81e844de admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:23:17.989377 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056cfdb5-5a9d-4cbe-927e-36cc81e844de admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:23:17.989945 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056cfdb5-5a9d-4cbe-927e-36cc81e844de admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:23:17.990693 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056cfdb5-5a9d-4cbe-927e-36cc81e844de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:23:17.992711 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056cfdb5-5a9d-4cbe-927e-36cc81e844de admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:23:18.029488 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1244/2473] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 09:23:17 2022] POST /identity/v3/projects => generated 379 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:23:18.030543 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d38bb1-2e2b-44d8-bd72-0ad7d4ef9952 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:23:18.042796 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d38bb1-2e2b-44d8-bd72-0ad7d4ef9952 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:23:18.111146 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50387b2-26a4-4f22-936e-2d9ea2427a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:23:18.112111 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50387b2-26a4-4f22-936e-2d9ea2427a10 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:23:18.112385 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50387b2-26a4-4f22-936e-2d9ea2427a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:23:18.112723 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50387b2-26a4-4f22-936e-2d9ea2427a10 admin admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:23:18.113424 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50387b2-26a4-4f22-936e-2d9ea2427a10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:23:18.115484 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50387b2-26a4-4f22-936e-2d9ea2427a10 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:23:18.149212 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1230/2474] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 09:23:18 2022] POST /identity/v3/users => generated 296 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 09:23:18.150521 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056cfdb5-5a9d-4cbe-927e-36cc81e844de admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:23:18.163071 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056cfdb5-5a9d-4cbe-927e-36cc81e844de admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:23:18.178331 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4ef965-b8a3-4b33-ad53-e9e01f5e6125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:23:18.179063 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ef965-b8a3-4b33-ad53-e9e01f5e6125 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:23:18.179413 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ef965-b8a3-4b33-ad53-e9e01f5e6125 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:23:18.179757 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ef965-b8a3-4b33-ad53-e9e01f5e6125 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:23:18.180479 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4ef965-b8a3-4b33-ad53-e9e01f5e6125 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:23:18.182201 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4ef965-b8a3-4b33-ad53-e9e01f5e6125 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:23:18.194015 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1245/2475] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:23:18 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 09:23:18.198151 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50387b2-26a4-4f22-936e-2d9ea2427a10 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:23:18.209474 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50387b2-26a4-4f22-936e-2d9ea2427a10 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:23:18.214491 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cfa559-0bd1-4312-8482-579f9141803d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:23:18.215277 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cfa559-0bd1-4312-8482-579f9141803d admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:23:18.215775 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cfa559-0bd1-4312-8482-579f9141803d admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:23:18.216275 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cfa559-0bd1-4312-8482-579f9141803d admin admin] PATH_INFO: `/v3/projects/0f0556a66f62422d877e74b0b30bd1c2/users/cdfdfa5581444cb7a27e693e5b622da8/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:23:18.233532 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cfa559-0bd1-4312-8482-579f9141803d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f0556a66f62422d877e74b0b30bd1c2, user_id=cdfdfa5581444cb7a27e693e5b622da8, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:23:18.236523 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cfa559-0bd1-4312-8482-579f9141803d admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:23:18.254531 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1231/2476] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:23:18 2022] PUT /identity/v3/projects/0f0556a66f62422d877e74b0b30bd1c2/users/cdfdfa5581444cb7a27e693e5b622da8/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:23:18.258246 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4ef965-b8a3-4b33-ad53-e9e01f5e6125 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:23:18.269990 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4ef965-b8a3-4b33-ad53-e9e01f5e6125 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:23:18.335560 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c0615b-5f17-491a-8aab-58b39ce73fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:23:18.336277 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c0615b-5f17-491a-8aab-58b39ce73fd6 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:23:18.336608 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c0615b-5f17-491a-8aab-58b39ce73fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:23:18.336885 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c0615b-5f17-491a-8aab-58b39ce73fd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:23:18.337580 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c0615b-5f17-491a-8aab-58b39ce73fd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:23:18.339421 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c0615b-5f17-491a-8aab-58b39ce73fd6 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:23:18.355139 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1246/2477] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:23:18 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 09:23:18.356340 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cfa559-0bd1-4312-8482-579f9141803d admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:23:18.367574 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cfa559-0bd1-4312-8482-579f9141803d admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:23:18.372812 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded1269b-683d-409e-b3f4-11d590c9615a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:23:18.373516 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded1269b-683d-409e-b3f4-11d590c9615a admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:23:18.373850 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded1269b-683d-409e-b3f4-11d590c9615a admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:23:18.374129 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded1269b-683d-409e-b3f4-11d590c9615a admin admin] PATH_INFO: `/v3/projects/0f0556a66f62422d877e74b0b30bd1c2/users/cdfdfa5581444cb7a27e693e5b622da8/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:23:18.376124 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded1269b-683d-409e-b3f4-11d590c9615a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f0556a66f62422d877e74b0b30bd1c2, user_id=cdfdfa5581444cb7a27e693e5b622da8, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:23:18.379622 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded1269b-683d-409e-b3f4-11d590c9615a admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:23:18.392497 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1232/2478] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:23:18 2022] PUT /identity/v3/projects/0f0556a66f62422d877e74b0b30bd1c2/users/cdfdfa5581444cb7a27e693e5b622da8/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:23:23.017402 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047789f8-569c-4cdc-8893-85818d9782cf None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:23:23.017963 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047789f8-569c-4cdc-8893-85818d9782cf None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:23:23.018082 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047789f8-569c-4cdc-8893-85818d9782cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:23:23.051782 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-047789f8-569c-4cdc-8893-85818d9782cf None None] MFA Rules not processed for user `cdfdfa5581444cb7a27e693e5b622da8`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:23:23.053023 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-047789f8-569c-4cdc-8893-85818d9782cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:23:23.092371 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:23:23.130341 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1247/2479] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:23:23 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:23:23.146052 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded1269b-683d-409e-b3f4-11d590c9615a admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:23:23.148383 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-ded1269b-683d-409e-b3f4-11d590c9615a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:23:23.161322 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:23:23.228126 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded1269b-683d-409e-b3f4-11d590c9615a admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:23:23.233703 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf82492-ff76-49da-b305-b9b85efd8d46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0a8af5cadc4cc1ab73e8eb2a20bb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:23:23.234425 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf82492-ff76-49da-b305-b9b85efd8d46 service nova] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:23:23.234780 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf82492-ff76-49da-b305-b9b85efd8d46 service nova] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:23:23.235060 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf82492-ff76-49da-b305-b9b85efd8d46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:23:23.292572 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf82492-ff76-49da-b305-b9b85efd8d46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:23:23.294207 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf82492-ff76-49da-b305-b9b85efd8d46 service nova] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:23:23.303951 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1233/2480] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:23:23 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 09:23:33.127123 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=13 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:23:33.299012 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=52 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:23:39.598694 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43f5a51-9c77-47a8-955d-7fe28be0edaf None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:23:39.599213 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43f5a51-9c77-47a8-955d-7fe28be0edaf None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:23:39.599321 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43f5a51-9c77-47a8-955d-7fe28be0edaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:23:39.618984 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-f43f5a51-9c77-47a8-955d-7fe28be0edaf None None] MFA Rules not processed for user `04b8dc0532de441db59bd4fe465a8c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:23:39.620176 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-f43f5a51-9c77-47a8-955d-7fe28be0edaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:23:39.626878 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1248/2481] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:23:39 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:23:41.384290 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf82492-ff76-49da-b305-b9b85efd8d46 service nova] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:23:41.389833 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-0cf82492-ff76-49da-b305-b9b85efd8d46 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:23:41.467020 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf82492-ff76-49da-b305-b9b85efd8d46 service nova] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:23:41.472035 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8140762c-5be0-44e8-ab90-f8abc49478db service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38986ca2612c4fc59d960cd693834f9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:23:41.472645 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8140762c-5be0-44e8-ab90-f8abc49478db service placement] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:23:41.472929 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8140762c-5be0-44e8-ab90-f8abc49478db service placement] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:23:41.473206 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8140762c-5be0-44e8-ab90-f8abc49478db service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:23:41.474940 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-8140762c-5be0-44e8-ab90-f8abc49478db service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:23:41.486760 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8140762c-5be0-44e8-ab90-f8abc49478db service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:23:41.488330 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8140762c-5be0-44e8-ab90-f8abc49478db service placement] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:23:41.498082 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1234/2482] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:23:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:23:43.183358 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43f5a51-9c77-47a8-955d-7fe28be0edaf None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:23:43.243490 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43f5a51-9c77-47a8-955d-7fe28be0edaf None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:23:43.247121 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b43847-27e5-4960-8b49-a9184a7edaf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:23:43.247672 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b43847-27e5-4960-8b49-a9184a7edaf5 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:23:43.247959 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b43847-27e5-4960-8b49-a9184a7edaf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:23:43.248236 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b43847-27e5-4960-8b49-a9184a7edaf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:23:43.249981 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-74b43847-27e5-4960-8b49-a9184a7edaf5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:23:43.260679 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b43847-27e5-4960-8b49-a9184a7edaf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:23:43.262157 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b43847-27e5-4960-8b49-a9184a7edaf5 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:23:43.270737 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1249/2483] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:23:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:23:51.493919 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=32 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:23:53.266646 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=29 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:23:54.112329 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8140762c-5be0-44e8-ab90-f8abc49478db service placement] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:23:54.114007 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-8140762c-5be0-44e8-ab90-f8abc49478db service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:23:54.143616 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8140762c-5be0-44e8-ab90-f8abc49478db service placement] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:23:54.151610 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f865af-56a1-4621-b1b3-2862496805a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0a8af5cadc4cc1ab73e8eb2a20bb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:23:54.153450 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f865af-56a1-4621-b1b3-2862496805a5 service nova] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:23:54.153845 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f865af-56a1-4621-b1b3-2862496805a5 service nova] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:23:54.154199 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f865af-56a1-4621-b1b3-2862496805a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:23:54.156169 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-e4f865af-56a1-4621-b1b3-2862496805a5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:23:54.170148 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f865af-56a1-4621-b1b3-2862496805a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:23:54.171894 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f865af-56a1-4621-b1b3-2862496805a5 service nova] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:23:54.198207 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1235/2484] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:23:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:23:56.807156 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b43847-27e5-4960-8b49-a9184a7edaf5 service neutron] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:23:56.808637 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-74b43847-27e5-4960-8b49-a9184a7edaf5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:23:56.840191 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b43847-27e5-4960-8b49-a9184a7edaf5 service neutron] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:23:56.843815 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0d7646-512e-461b-ae1a-b7cfc6f66e50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:23:56.844415 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0d7646-512e-461b-ae1a-b7cfc6f66e50 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:23:56.844692 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0d7646-512e-461b-ae1a-b7cfc6f66e50 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:23:56.844963 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0d7646-512e-461b-ae1a-b7cfc6f66e50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:23:56.872557 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0d7646-512e-461b-ae1a-b7cfc6f66e50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:23:56.874002 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0d7646-512e-461b-ae1a-b7cfc6f66e50 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:23:56.883608 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1250/2485] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:23:56 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:24:04.179644 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=6 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:24:05.916403 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f865af-56a1-4621-b1b3-2862496805a5 service nova] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:05.965413 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f865af-56a1-4621-b1b3-2862496805a5 service nova] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:05.970358 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563c961e-aa33-4a51-b6a1-49960ca42f11 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38986ca2612c4fc59d960cd693834f9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:05.970952 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c961e-aa33-4a51-b6a1-49960ca42f11 service placement] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:05.971231 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c961e-aa33-4a51-b6a1-49960ca42f11 service placement] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:05.971497 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c961e-aa33-4a51-b6a1-49960ca42f11 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:05.973192 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-563c961e-aa33-4a51-b6a1-49960ca42f11 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:24:05.997121 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563c961e-aa33-4a51-b6a1-49960ca42f11 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:05.998744 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563c961e-aa33-4a51-b6a1-49960ca42f11 service placement] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:06.008550 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1236/2486] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:24:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:24:06.878976 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=23 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:24:16.004276 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=27 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:24:19.855172 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0d7646-512e-461b-ae1a-b7cfc6f66e50 service neutron] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:19.863769 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0d7646-512e-461b-ae1a-b7cfc6f66e50 service neutron] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:19.867535 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5510e9d1-6435-4961-8e39-18cdbb9d0a95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:19.868130 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510e9d1-6435-4961-8e39-18cdbb9d0a95 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:19.868410 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510e9d1-6435-4961-8e39-18cdbb9d0a95 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:19.868685 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510e9d1-6435-4961-8e39-18cdbb9d0a95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:19.870408 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-5510e9d1-6435-4961-8e39-18cdbb9d0a95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:24:19.932549 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5510e9d1-6435-4961-8e39-18cdbb9d0a95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:19.933984 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5510e9d1-6435-4961-8e39-18cdbb9d0a95 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:19.943455 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1251/2487] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:24:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:24:22.819204 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563c961e-aa33-4a51-b6a1-49960ca42f11 service placement] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:22.821017 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-563c961e-aa33-4a51-b6a1-49960ca42f11 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:24:22.835844 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563c961e-aa33-4a51-b6a1-49960ca42f11 service placement] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:22.840463 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd96bcb-96d7-4eac-a53b-4e4e8d74d052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:22.841049 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd96bcb-96d7-4eac-a53b-4e4e8d74d052 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:22.841326 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd96bcb-96d7-4eac-a53b-4e4e8d74d052 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:22.841598 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd96bcb-96d7-4eac-a53b-4e4e8d74d052 admin admin] PATH_INFO: `/v3/users/a5d85f7efca040cc81253dfbf0de7c42` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:22.842796 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd96bcb-96d7-4eac-a53b-4e4e8d74d052 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5d85f7efca040cc81253dfbf0de7c42)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:22.844744 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd96bcb-96d7-4eac-a53b-4e4e8d74d052 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:22.898617 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-4fd96bcb-96d7-4eac-a53b-4e4e8d74d052 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5d85f7efca040cc81253dfbf0de7c42'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:22.899281 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-4fd96bcb-96d7-4eac-a53b-4e4e8d74d052 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5d85f7efca040cc81253dfbf0de7c42'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:22.910942 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-4fd96bcb-96d7-4eac-a53b-4e4e8d74d052 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5d85f7efca040cc81253dfbf0de7c42'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:22.918672 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-4fd96bcb-96d7-4eac-a53b-4e4e8d74d052 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5d85f7efca040cc81253dfbf0de7c42'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:22.919282 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-4fd96bcb-96d7-4eac-a53b-4e4e8d74d052 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5d85f7efca040cc81253dfbf0de7c42'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:22.932370 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1237/2488] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 09:24:22 2022] DELETE /identity/v3/users/a5d85f7efca040cc81253dfbf0de7c42 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:24:23.096690 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5510e9d1-6435-4961-8e39-18cdbb9d0a95 service neutron] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:23.107323 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5510e9d1-6435-4961-8e39-18cdbb9d0a95 service neutron] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:23.163465 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd188ee-b831-4429-b6b3-a4e4c0221131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:23.164544 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd188ee-b831-4429-b6b3-a4e4c0221131 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:23.164823 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd188ee-b831-4429-b6b3-a4e4c0221131 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:23.165096 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd188ee-b831-4429-b6b3-a4e4c0221131 admin admin] PATH_INFO: `/v3/projects/becb9cd46453447a87846cf26561a190` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:23.166222 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd188ee-b831-4429-b6b3-a4e4c0221131 admin admin] RBAC: Authorizing `identity:delete_project(project_id=becb9cd46453447a87846cf26561a190)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:23.168115 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd188ee-b831-4429-b6b3-a4e4c0221131 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:23.212321 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-ffd188ee-b831-4429-b6b3-a4e4c0221131 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'becb9cd46453447a87846cf26561a190', 'request_id': 'req-ffd188ee-b831-4429-b6b3-a4e4c0221131'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:23.218391 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-ffd188ee-b831-4429-b6b3-a4e4c0221131 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'becb9cd46453447a87846cf26561a190', 'request_id': 'req-ffd188ee-b831-4429-b6b3-a4e4c0221131'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:23.225803 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-ffd188ee-b831-4429-b6b3-a4e4c0221131 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'becb9cd46453447a87846cf26561a190', 'request_id': 'req-ffd188ee-b831-4429-b6b3-a4e4c0221131'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:23.236743 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-ffd188ee-b831-4429-b6b3-a4e4c0221131 admin admin] The token cache is being invalidate because project becb9cd46453447a87846cf26561a190 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 09:24:23.237035 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-ffd188ee-b831-4429-b6b3-a4e4c0221131 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:23.237658 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-ffd188ee-b831-4429-b6b3-a4e4c0221131 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:23.239157 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1252/2489] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 09:24:23 2022] DELETE /identity/v3/projects/becb9cd46453447a87846cf26561a190 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:24:23.244934 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f125d92f-41b9-4a4d-80e7-6772172d10bb None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:23.245297 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f125d92f-41b9-4a4d-80e7-6772172d10bb None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:23.245646 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f125d92f-41b9-4a4d-80e7-6772172d10bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:23.265123 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-f125d92f-41b9-4a4d-80e7-6772172d10bb None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:24:23.266360 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-f125d92f-41b9-4a4d-80e7-6772172d10bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:24:23.324634 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1238/2490] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:24:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:24:23.332685 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05a3abb-5cc0-4528-b96b-8589f1db4fe6 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:23.332976 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05a3abb-5cc0-4528-b96b-8589f1db4fe6 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:23.333253 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05a3abb-5cc0-4528-b96b-8589f1db4fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:23.353633 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-c05a3abb-5cc0-4528-b96b-8589f1db4fe6 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:24:23.354876 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-c05a3abb-5cc0-4528-b96b-8589f1db4fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:24:23.362688 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1253/2491] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:24:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:24:23.366747 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f125d92f-41b9-4a4d-80e7-6772172d10bb None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:23.376239 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f125d92f-41b9-4a4d-80e7-6772172d10bb None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:23.380768 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9eb548-dbca-4da1-96df-fcf6dc102dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:23.381354 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9eb548-dbca-4da1-96df-fcf6dc102dfb admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:23.381631 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9eb548-dbca-4da1-96df-fcf6dc102dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:23.381948 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9eb548-dbca-4da1-96df-fcf6dc102dfb admin admin] PATH_INFO: `/v3/domains` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:23.382589 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9eb548-dbca-4da1-96df-fcf6dc102dfb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 09:24:23.382891 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9eb548-dbca-4da1-96df-fcf6dc102dfb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:23.384495 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9eb548-dbca-4da1-96df-fcf6dc102dfb admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:23.396179 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1239/2492] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 09:24:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 09:24:23.401020 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05a3abb-5cc0-4528-b96b-8589f1db4fe6 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:23.409992 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05a3abb-5cc0-4528-b96b-8589f1db4fe6 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:23.415198 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c6391e-0056-4c03-8558-9699fdf64a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:23.415730 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c6391e-0056-4c03-8558-9699fdf64a76 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:23.416006 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c6391e-0056-4c03-8558-9699fdf64a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:23.416275 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c6391e-0056-4c03-8558-9699fdf64a76 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:23.416906 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c6391e-0056-4c03-8558-9699fdf64a76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:23.418823 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c6391e-0056-4c03-8558-9699fdf64a76 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:23.434300 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:24:23.450242 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1254/2493] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 09:24:23 2022] POST /identity/v3/projects => generated 401 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:24:23.451373 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9eb548-dbca-4da1-96df-fcf6dc102dfb admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:23.461816 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9eb548-dbca-4da1-96df-fcf6dc102dfb admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:23.517096 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de76ba7f-dd7a-4b6e-8c9c-e2e024ec0057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:23.517691 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76ba7f-dd7a-4b6e-8c9c-e2e024ec0057 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:23.517991 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76ba7f-dd7a-4b6e-8c9c-e2e024ec0057 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:23.518268 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76ba7f-dd7a-4b6e-8c9c-e2e024ec0057 admin admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:23.518899 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de76ba7f-dd7a-4b6e-8c9c-e2e024ec0057 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:23.520821 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de76ba7f-dd7a-4b6e-8c9c-e2e024ec0057 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:23.529269 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:24:23.543755 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1240/2494] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 09:24:23 2022] POST /identity/v3/users => generated 301 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:24:23.547625 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c6391e-0056-4c03-8558-9699fdf64a76 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:23.558035 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c6391e-0056-4c03-8558-9699fdf64a76 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:23.562574 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88b9327-c612-4750-bbd1-f344ef94de9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:23.563165 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b9327-c612-4750-bbd1-f344ef94de9a admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:23.563443 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b9327-c612-4750-bbd1-f344ef94de9a admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:23.563712 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b9327-c612-4750-bbd1-f344ef94de9a admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:23.564340 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88b9327-c612-4750-bbd1-f344ef94de9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:23.565940 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88b9327-c612-4750-bbd1-f344ef94de9a admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:23.576195 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1255/2495] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:24:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:24:23.579962 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de76ba7f-dd7a-4b6e-8c9c-e2e024ec0057 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:23.590318 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de76ba7f-dd7a-4b6e-8c9c-e2e024ec0057 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:23.594840 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c560b139-991a-44ae-8002-1a555cf07f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:23.595426 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560b139-991a-44ae-8002-1a555cf07f53 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:23.595703 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560b139-991a-44ae-8002-1a555cf07f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:23.595972 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560b139-991a-44ae-8002-1a555cf07f53 admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:23.596603 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c560b139-991a-44ae-8002-1a555cf07f53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:23.598247 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c560b139-991a-44ae-8002-1a555cf07f53 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:23.608497 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1241/2496] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:24:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:24:23.612594 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88b9327-c612-4750-bbd1-f344ef94de9a admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:23.622265 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88b9327-c612-4750-bbd1-f344ef94de9a admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:23.626788 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8369221d-d580-4285-95ca-9d1b308e7e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:23.627419 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8369221d-d580-4285-95ca-9d1b308e7e57 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:23.627712 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8369221d-d580-4285-95ca-9d1b308e7e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:23.627972 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8369221d-d580-4285-95ca-9d1b308e7e57 admin admin] PATH_INFO: `/v3/projects/69eed12b39634d2b996526772c13abbc/users/65081e317e2c4827862749ee534518f9/roles/3248f0268738489b883041851d681bb7` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:23.643915 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8369221d-d580-4285-95ca-9d1b308e7e57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69eed12b39634d2b996526772c13abbc, user_id=65081e317e2c4827862749ee534518f9, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:23.646811 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8369221d-d580-4285-95ca-9d1b308e7e57 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:23.660515 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1256/2497] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:24:23 2022] PUT /identity/v3/projects/69eed12b39634d2b996526772c13abbc/users/65081e317e2c4827862749ee534518f9/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 09:24:23.670509 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af3897d-4bb3-4627-8958-01964867edd7 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:23.670826 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af3897d-4bb3-4627-8958-01964867edd7 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:23.671207 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af3897d-4bb3-4627-8958-01964867edd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:23.699070 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-6af3897d-4bb3-4627-8958-01964867edd7 None None] MFA Rules not processed for user `65081e317e2c4827862749ee534518f9`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:24:23.700265 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-6af3897d-4bb3-4627-8958-01964867edd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:24:23.763088 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1242/2498] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:24:23 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:24:23.767641 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8369221d-d580-4285-95ca-9d1b308e7e57 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:23.828294 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8369221d-d580-4285-95ca-9d1b308e7e57 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:23.832879 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80d7d5e-abe2-452b-979b-2df213e4184f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:23.833474 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80d7d5e-abe2-452b-979b-2df213e4184f admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:23.833783 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80d7d5e-abe2-452b-979b-2df213e4184f admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:23.834061 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80d7d5e-abe2-452b-979b-2df213e4184f admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:23.834682 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80d7d5e-abe2-452b-979b-2df213e4184f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:23.836585 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80d7d5e-abe2-452b-979b-2df213e4184f admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:23.862318 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1257/2499] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 09:24:23 2022] POST /identity/v3/projects => generated 399 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:24:23.866393 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af3897d-4bb3-4627-8958-01964867edd7 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:23.877606 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af3897d-4bb3-4627-8958-01964867edd7 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:23.933341 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abe887e-1637-4807-acfc-da2476f4dca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:23.933910 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abe887e-1637-4807-acfc-da2476f4dca2 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:23.934196 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abe887e-1637-4807-acfc-da2476f4dca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:23.934470 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abe887e-1637-4807-acfc-da2476f4dca2 admin admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:23.935090 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abe887e-1637-4807-acfc-da2476f4dca2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:23.937009 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abe887e-1637-4807-acfc-da2476f4dca2 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:23.957071 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1243/2500] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 09:24:23 2022] POST /identity/v3/users => generated 306 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:24:23.961661 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80d7d5e-abe2-452b-979b-2df213e4184f admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:23.971880 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80d7d5e-abe2-452b-979b-2df213e4184f admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:23.977077 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227aa0a2-943a-4949-8e02-5fa41af25021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:23.977672 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227aa0a2-943a-4949-8e02-5fa41af25021 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:23.977982 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227aa0a2-943a-4949-8e02-5fa41af25021 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:23.978261 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227aa0a2-943a-4949-8e02-5fa41af25021 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:23.978899 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227aa0a2-943a-4949-8e02-5fa41af25021 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:23.980493 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227aa0a2-943a-4949-8e02-5fa41af25021 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:23.990885 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1258/2501] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:24:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:24:23.994828 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abe887e-1637-4807-acfc-da2476f4dca2 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:24.004585 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abe887e-1637-4807-acfc-da2476f4dca2 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:24.009141 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1195b5-3907-44e3-99a2-781f42fe0e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:24.009809 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1195b5-3907-44e3-99a2-781f42fe0e6f admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:24.010097 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1195b5-3907-44e3-99a2-781f42fe0e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:24.010375 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1195b5-3907-44e3-99a2-781f42fe0e6f admin admin] PATH_INFO: `/v3/projects/880d580efb8e4061b41595355d80690f/users/215311f5ffcd45de92d468b4cf4d3f19/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:24.025136 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1195b5-3907-44e3-99a2-781f42fe0e6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=880d580efb8e4061b41595355d80690f, user_id=215311f5ffcd45de92d468b4cf4d3f19, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:24.027812 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1195b5-3907-44e3-99a2-781f42fe0e6f admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:24.040999 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1244/2502] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:24:23 2022] PUT /identity/v3/projects/880d580efb8e4061b41595355d80690f/users/215311f5ffcd45de92d468b4cf4d3f19/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:24:24.044965 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227aa0a2-943a-4949-8e02-5fa41af25021 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:24.055270 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227aa0a2-943a-4949-8e02-5fa41af25021 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:24.114580 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199feda7-917a-47a7-9cd2-3d88e57f55ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:24.115190 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199feda7-917a-47a7-9cd2-3d88e57f55ad admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:24.115470 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199feda7-917a-47a7-9cd2-3d88e57f55ad admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:24.115766 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199feda7-917a-47a7-9cd2-3d88e57f55ad admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:24.116403 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199feda7-917a-47a7-9cd2-3d88e57f55ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:24.118061 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199feda7-917a-47a7-9cd2-3d88e57f55ad admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:24.128054 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1259/2503] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:24:24 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:24:24.132171 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1195b5-3907-44e3-99a2-781f42fe0e6f admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:24.142517 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1195b5-3907-44e3-99a2-781f42fe0e6f admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:24.147136 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77da8087-683a-4bab-ae20-307d01706475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:24.147788 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da8087-683a-4bab-ae20-307d01706475 admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:24.148668 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da8087-683a-4bab-ae20-307d01706475 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:24.148947 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da8087-683a-4bab-ae20-307d01706475 admin admin] PATH_INFO: `/v3/projects/880d580efb8e4061b41595355d80690f/users/215311f5ffcd45de92d468b4cf4d3f19/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:24.150887 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77da8087-683a-4bab-ae20-307d01706475 admin admin] RBAC: Authorizing `identity:create_grant(project_id=880d580efb8e4061b41595355d80690f, user_id=215311f5ffcd45de92d468b4cf4d3f19, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:24.153513 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77da8087-683a-4bab-ae20-307d01706475 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:24.164722 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1245/2504] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:24:24 2022] PUT /identity/v3/projects/880d580efb8e4061b41595355d80690f/users/215311f5ffcd45de92d468b4cf4d3f19/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:24:24.176002 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e87286-688f-41f9-81b5-d14a14275310 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:24.176287 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e87286-688f-41f9-81b5-d14a14275310 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:24.176564 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e87286-688f-41f9-81b5-d14a14275310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:24.203843 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-c4e87286-688f-41f9-81b5-d14a14275310 None None] MFA Rules not processed for user `215311f5ffcd45de92d468b4cf4d3f19`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:24:24.204975 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-c4e87286-688f-41f9-81b5-d14a14275310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:24:24.223113 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:24:24.274004 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1260/2505] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:24:24 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:24:24.284566 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77da8087-683a-4bab-ae20-307d01706475 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:24.286316 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-77da8087-683a-4bab-ae20-307d01706475 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:24:24.309093 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:24:24.345087 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77da8087-683a-4bab-ae20-307d01706475 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:24.348796 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffe07ae-7095-4022-8f12-051ba78aace7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:24.349398 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffe07ae-7095-4022-8f12-051ba78aace7 service cinder] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:24.349700 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffe07ae-7095-4022-8f12-051ba78aace7 service cinder] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:24.349981 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffe07ae-7095-4022-8f12-051ba78aace7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:24.358954 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffe07ae-7095-4022-8f12-051ba78aace7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:24.360362 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffe07ae-7095-4022-8f12-051ba78aace7 service cinder] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:24.369661 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1246/2506] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:24:24 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:24:24.449551 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e87286-688f-41f9-81b5-d14a14275310 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:24.459243 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e87286-688f-41f9-81b5-d14a14275310 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:24.463588 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb52ba2b-6a18-4e05-86e9-7a2d93f0d50f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:24.464197 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb52ba2b-6a18-4e05-86e9-7a2d93f0d50f service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:24.464498 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb52ba2b-6a18-4e05-86e9-7a2d93f0d50f service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:24.464765 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb52ba2b-6a18-4e05-86e9-7a2d93f0d50f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:24.520058 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb52ba2b-6a18-4e05-86e9-7a2d93f0d50f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:24.521478 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb52ba2b-6a18-4e05-86e9-7a2d93f0d50f service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:24.530896 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1261/2507] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:24:24 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:24:28.572884 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffe07ae-7095-4022-8f12-051ba78aace7 service cinder] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:28.634868 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffe07ae-7095-4022-8f12-051ba78aace7 service cinder] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:28.639270 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e830af-3701-4902-ac21-ae9cfb0599d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:28.639889 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e830af-3701-4902-ac21-ae9cfb0599d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:28.640169 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e830af-3701-4902-ac21-ae9cfb0599d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:28.640440 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e830af-3701-4902-ac21-ae9cfb0599d9 admin admin] PATH_INFO: `/v3/users/65081e317e2c4827862749ee534518f9` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:28.641521 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e830af-3701-4902-ac21-ae9cfb0599d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65081e317e2c4827862749ee534518f9)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:28.643442 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e830af-3701-4902-ac21-ae9cfb0599d9 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:28.690892 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-c4e830af-3701-4902-ac21-ae9cfb0599d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65081e317e2c4827862749ee534518f9'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:28.691587 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-c4e830af-3701-4902-ac21-ae9cfb0599d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65081e317e2c4827862749ee534518f9'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:28.703774 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-c4e830af-3701-4902-ac21-ae9cfb0599d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65081e317e2c4827862749ee534518f9'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:28.710651 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-c4e830af-3701-4902-ac21-ae9cfb0599d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65081e317e2c4827862749ee534518f9'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:28.711284 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-c4e830af-3701-4902-ac21-ae9cfb0599d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65081e317e2c4827862749ee534518f9'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:28.723133 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1247/2508] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 09:24:28 2022] DELETE /identity/v3/users/65081e317e2c4827862749ee534518f9 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:24:28.728162 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10978a79-81d7-485f-8f74-151b96caa73e None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:28.728611 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10978a79-81d7-485f-8f74-151b96caa73e None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:28.728741 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10978a79-81d7-485f-8f74-151b96caa73e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:28.748043 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-10978a79-81d7-485f-8f74-151b96caa73e None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:24:28.749246 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-10978a79-81d7-485f-8f74-151b96caa73e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:24:28.806354 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1262/2509] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:24:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:24:28.816114 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e830af-3701-4902-ac21-ae9cfb0599d9 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:28.817477 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-c4e830af-3701-4902-ac21-ae9cfb0599d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:24:28.861434 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:24:28.874653 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e830af-3701-4902-ac21-ae9cfb0599d9 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:28.878225 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da07840b-8905-4581-8ccd-63bbce163a8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:28.878820 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da07840b-8905-4581-8ccd-63bbce163a8d service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:28.879094 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da07840b-8905-4581-8ccd-63bbce163a8d service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:28.879364 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da07840b-8905-4581-8ccd-63bbce163a8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:28.888486 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da07840b-8905-4581-8ccd-63bbce163a8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:28.889975 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da07840b-8905-4581-8ccd-63bbce163a8d service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:28.899270 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1248/2510] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:24:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:24:29.155118 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10978a79-81d7-485f-8f74-151b96caa73e None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:29.156577 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-10978a79-81d7-485f-8f74-151b96caa73e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:24:29.168915 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10978a79-81d7-485f-8f74-151b96caa73e None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:29.173989 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407f0d42-4144-48e4-b2fd-c47a866edcd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:29.174619 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f0d42-4144-48e4-b2fd-c47a866edcd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:29.174902 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f0d42-4144-48e4-b2fd-c47a866edcd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:29.175285 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f0d42-4144-48e4-b2fd-c47a866edcd1 admin admin] PATH_INFO: `/v3/projects/69eed12b39634d2b996526772c13abbc` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:29.176349 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407f0d42-4144-48e4-b2fd-c47a866edcd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69eed12b39634d2b996526772c13abbc)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:29.178542 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407f0d42-4144-48e4-b2fd-c47a866edcd1 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:29.224134 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-407f0d42-4144-48e4-b2fd-c47a866edcd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69eed12b39634d2b996526772c13abbc', 'request_id': 'req-407f0d42-4144-48e4-b2fd-c47a866edcd1'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:29.225213 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:24:29.232481 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-407f0d42-4144-48e4-b2fd-c47a866edcd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69eed12b39634d2b996526772c13abbc', 'request_id': 'req-407f0d42-4144-48e4-b2fd-c47a866edcd1'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:29.239785 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-407f0d42-4144-48e4-b2fd-c47a866edcd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69eed12b39634d2b996526772c13abbc', 'request_id': 'req-407f0d42-4144-48e4-b2fd-c47a866edcd1'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:29.249898 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-407f0d42-4144-48e4-b2fd-c47a866edcd1 admin admin] The token cache is being invalidate because project 69eed12b39634d2b996526772c13abbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 09:24:29.250178 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-407f0d42-4144-48e4-b2fd-c47a866edcd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:29.250799 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-407f0d42-4144-48e4-b2fd-c47a866edcd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:29.252270 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1263/2511] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 09:24:29 2022] DELETE /identity/v3/projects/69eed12b39634d2b996526772c13abbc => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:24:29.256229 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da07840b-8905-4581-8ccd-63bbce163a8d service neutron] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:29.265828 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da07840b-8905-4581-8ccd-63bbce163a8d service neutron] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:29.321494 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24106450-379f-49af-950d-4796570dbe34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:29.322142 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24106450-379f-49af-950d-4796570dbe34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:29.322418 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24106450-379f-49af-950d-4796570dbe34 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:29.322688 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24106450-379f-49af-950d-4796570dbe34 admin admin] PATH_INFO: `/v3/users/215311f5ffcd45de92d468b4cf4d3f19` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:29.323782 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24106450-379f-49af-950d-4796570dbe34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=215311f5ffcd45de92d468b4cf4d3f19)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:29.325697 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24106450-379f-49af-950d-4796570dbe34 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:29.371979 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-24106450-379f-49af-950d-4796570dbe34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '215311f5ffcd45de92d468b4cf4d3f19'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:29.372625 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-24106450-379f-49af-950d-4796570dbe34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '215311f5ffcd45de92d468b4cf4d3f19'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:29.384641 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-24106450-379f-49af-950d-4796570dbe34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '215311f5ffcd45de92d468b4cf4d3f19'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:29.391262 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-24106450-379f-49af-950d-4796570dbe34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '215311f5ffcd45de92d468b4cf4d3f19'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:29.391882 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-24106450-379f-49af-950d-4796570dbe34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '215311f5ffcd45de92d468b4cf4d3f19'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:29.405010 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1249/2512] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 09:24:29 2022] DELETE /identity/v3/users/215311f5ffcd45de92d468b4cf4d3f19 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:24:29.940491 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407f0d42-4144-48e4-b2fd-c47a866edcd1 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:29.941968 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-407f0d42-4144-48e4-b2fd-c47a866edcd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:24:29.954645 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407f0d42-4144-48e4-b2fd-c47a866edcd1 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:30.011082 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9585f8-54d8-4b6c-982e-d5517c45bbb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:30.011719 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9585f8-54d8-4b6c-982e-d5517c45bbb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:30.012000 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9585f8-54d8-4b6c-982e-d5517c45bbb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:30.012275 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9585f8-54d8-4b6c-982e-d5517c45bbb5 admin admin] PATH_INFO: `/v3/projects/880d580efb8e4061b41595355d80690f` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:30.013377 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9585f8-54d8-4b6c-982e-d5517c45bbb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=880d580efb8e4061b41595355d80690f)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:30.015297 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9585f8-54d8-4b6c-982e-d5517c45bbb5 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:30.060900 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-9b9585f8-54d8-4b6c-982e-d5517c45bbb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '880d580efb8e4061b41595355d80690f', 'request_id': 'req-9b9585f8-54d8-4b6c-982e-d5517c45bbb5'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:30.067089 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-9b9585f8-54d8-4b6c-982e-d5517c45bbb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '880d580efb8e4061b41595355d80690f', 'request_id': 'req-9b9585f8-54d8-4b6c-982e-d5517c45bbb5'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:30.074476 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-9b9585f8-54d8-4b6c-982e-d5517c45bbb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '880d580efb8e4061b41595355d80690f', 'request_id': 'req-9b9585f8-54d8-4b6c-982e-d5517c45bbb5'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:30.086530 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-9b9585f8-54d8-4b6c-982e-d5517c45bbb5 admin admin] The token cache is being invalidate because project 880d580efb8e4061b41595355d80690f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 09:24:30.086825 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-9b9585f8-54d8-4b6c-982e-d5517c45bbb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:30.087485 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-9b9585f8-54d8-4b6c-982e-d5517c45bbb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:30.089146 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1264/2513] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 09:24:29 2022] DELETE /identity/v3/projects/880d580efb8e4061b41595355d80690f => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:24:30.094294 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc666d5-bc6d-4f28-a1f7-c97c456bc3d6 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:30.094800 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc666d5-bc6d-4f28-a1f7-c97c456bc3d6 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:30.095316 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc666d5-bc6d-4f28-a1f7-c97c456bc3d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:30.115471 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-bfc666d5-bc6d-4f28-a1f7-c97c456bc3d6 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:24:30.116696 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-bfc666d5-bc6d-4f28-a1f7-c97c456bc3d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:24:30.174154 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1250/2514] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:24:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:24:30.181988 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a384180-dc60-4ff4-8701-e2d62461490e None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:30.182273 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a384180-dc60-4ff4-8701-e2d62461490e None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:30.182546 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a384180-dc60-4ff4-8701-e2d62461490e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:30.203366 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-5a384180-dc60-4ff4-8701-e2d62461490e None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:24:30.204616 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-5a384180-dc60-4ff4-8701-e2d62461490e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:24:30.213533 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1265/2515] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:24:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:24:30.217902 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc666d5-bc6d-4f28-a1f7-c97c456bc3d6 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:30.223942 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:24:30.228411 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc666d5-bc6d-4f28-a1f7-c97c456bc3d6 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:30.234190 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2d6283-e346-4ea7-bd2e-cabd26623591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:30.234987 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d6283-e346-4ea7-bd2e-cabd26623591 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:30.235486 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d6283-e346-4ea7-bd2e-cabd26623591 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:30.235855 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d6283-e346-4ea7-bd2e-cabd26623591 admin admin] PATH_INFO: `/v3/domains` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:30.236684 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2d6283-e346-4ea7-bd2e-cabd26623591 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 09:24:30.236994 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2d6283-e346-4ea7-bd2e-cabd26623591 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:30.238675 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2d6283-e346-4ea7-bd2e-cabd26623591 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:30.250133 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1251/2516] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 09:24:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 09:24:30.254500 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a384180-dc60-4ff4-8701-e2d62461490e None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:30.264179 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a384180-dc60-4ff4-8701-e2d62461490e None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:30.269274 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc458235-c45a-4bdd-a482-14efb6824710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:30.269901 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc458235-c45a-4bdd-a482-14efb6824710 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:30.270214 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc458235-c45a-4bdd-a482-14efb6824710 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:30.270494 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc458235-c45a-4bdd-a482-14efb6824710 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:30.271137 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc458235-c45a-4bdd-a482-14efb6824710 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:30.273651 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc458235-c45a-4bdd-a482-14efb6824710 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:30.300261 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1266/2517] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 09:24:30 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 09:24:30.304558 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2d6283-e346-4ea7-bd2e-cabd26623591 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:30.314316 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2d6283-e346-4ea7-bd2e-cabd26623591 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:30.369821 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8624d83-bfe8-4f8a-80d1-8b329b8642fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:30.370431 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8624d83-bfe8-4f8a-80d1-8b329b8642fa admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:30.370716 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8624d83-bfe8-4f8a-80d1-8b329b8642fa admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:30.370996 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8624d83-bfe8-4f8a-80d1-8b329b8642fa admin admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:30.371627 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8624d83-bfe8-4f8a-80d1-8b329b8642fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:30.373550 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8624d83-bfe8-4f8a-80d1-8b329b8642fa admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:30.398920 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1252/2518] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 09:24:30 2022] POST /identity/v3/users => generated 290 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:24:30.400208 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc458235-c45a-4bdd-a482-14efb6824710 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:30.410487 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc458235-c45a-4bdd-a482-14efb6824710 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:30.415074 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4ad9ca-adbe-4941-8f46-4832f145b4c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:30.415668 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4ad9ca-adbe-4941-8f46-4832f145b4c2 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:30.415953 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4ad9ca-adbe-4941-8f46-4832f145b4c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:30.416231 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4ad9ca-adbe-4941-8f46-4832f145b4c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:30.416891 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4ad9ca-adbe-4941-8f46-4832f145b4c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:30.418528 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4ad9ca-adbe-4941-8f46-4832f145b4c2 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:30.429009 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1267/2519] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:24:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 09:24:30.432741 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8624d83-bfe8-4f8a-80d1-8b329b8642fa admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:30.442426 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8624d83-bfe8-4f8a-80d1-8b329b8642fa admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:30.447439 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440d9642-96be-4f8b-8cdb-75f8a7c802d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:30.448021 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440d9642-96be-4f8b-8cdb-75f8a7c802d6 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:30.448304 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440d9642-96be-4f8b-8cdb-75f8a7c802d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:30.448573 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440d9642-96be-4f8b-8cdb-75f8a7c802d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:30.449202 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440d9642-96be-4f8b-8cdb-75f8a7c802d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:30.450824 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440d9642-96be-4f8b-8cdb-75f8a7c802d6 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:30.462104 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1253/2520] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:24:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:24:30.466090 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4ad9ca-adbe-4941-8f46-4832f145b4c2 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:30.475896 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4ad9ca-adbe-4941-8f46-4832f145b4c2 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:30.480558 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0304b9-62a7-4301-8f08-e7a08cc78611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:30.481203 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0304b9-62a7-4301-8f08-e7a08cc78611 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:30.481556 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0304b9-62a7-4301-8f08-e7a08cc78611 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:30.481803 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0304b9-62a7-4301-8f08-e7a08cc78611 admin admin] PATH_INFO: `/v3/projects/a7741fcacd2f41e6b4517552e233e12d/users/d0d4611b152a4a1c9057de2b37a10dd9/roles/3248f0268738489b883041851d681bb7` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:30.496399 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0304b9-62a7-4301-8f08-e7a08cc78611 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7741fcacd2f41e6b4517552e233e12d, user_id=d0d4611b152a4a1c9057de2b37a10dd9, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:30.499052 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0304b9-62a7-4301-8f08-e7a08cc78611 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:30.511637 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1268/2521] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:24:30 2022] PUT /identity/v3/projects/a7741fcacd2f41e6b4517552e233e12d/users/d0d4611b152a4a1c9057de2b37a10dd9/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 09:24:30.522632 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e61fd-c84a-4cce-b21c-a32505eede41 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:30.523535 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e61fd-c84a-4cce-b21c-a32505eede41 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:30.523812 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e61fd-c84a-4cce-b21c-a32505eede41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:30.551168 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-885e61fd-c84a-4cce-b21c-a32505eede41 None None] MFA Rules not processed for user `d0d4611b152a4a1c9057de2b37a10dd9`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:24:30.552646 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-885e61fd-c84a-4cce-b21c-a32505eede41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:24:30.614921 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1254/2522] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:24:30 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:24:30.619782 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0304b9-62a7-4301-8f08-e7a08cc78611 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:30.625324 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:24:30.683155 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0304b9-62a7-4301-8f08-e7a08cc78611 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:30.687740 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64b0ee4-e5c4-4131-9c4d-b717bc0f40e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:30.688337 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64b0ee4-e5c4-4131-9c4d-b717bc0f40e8 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:30.688619 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64b0ee4-e5c4-4131-9c4d-b717bc0f40e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:30.688902 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64b0ee4-e5c4-4131-9c4d-b717bc0f40e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:30.689517 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64b0ee4-e5c4-4131-9c4d-b717bc0f40e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:30.691455 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64b0ee4-e5c4-4131-9c4d-b717bc0f40e8 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:30.717496 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1269/2523] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 09:24:30 2022] POST /identity/v3/projects => generated 379 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:24:30.721450 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885e61fd-c84a-4cce-b21c-a32505eede41 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:30.732136 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885e61fd-c84a-4cce-b21c-a32505eede41 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:30.787648 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee94698-787a-4bf1-a79b-d1ba4b24f656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:30.788269 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee94698-787a-4bf1-a79b-d1ba4b24f656 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:30.788553 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee94698-787a-4bf1-a79b-d1ba4b24f656 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:30.788829 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee94698-787a-4bf1-a79b-d1ba4b24f656 admin admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:30.789479 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee94698-787a-4bf1-a79b-d1ba4b24f656 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:30.791434 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee94698-787a-4bf1-a79b-d1ba4b24f656 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:30.799223 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:24:30.813688 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1255/2524] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 09:24:30 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 09:24:30.817586 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64b0ee4-e5c4-4131-9c4d-b717bc0f40e8 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:30.828426 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64b0ee4-e5c4-4131-9c4d-b717bc0f40e8 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:30.833056 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3a3b72-b2f4-4b43-8ef7-569bcb07775d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:30.833653 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a3b72-b2f4-4b43-8ef7-569bcb07775d admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:30.833977 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a3b72-b2f4-4b43-8ef7-569bcb07775d admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:30.834254 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a3b72-b2f4-4b43-8ef7-569bcb07775d admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:30.834893 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3a3b72-b2f4-4b43-8ef7-569bcb07775d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:30.836479 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3a3b72-b2f4-4b43-8ef7-569bcb07775d admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:30.846804 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1270/2525] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:24:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:24:30.850449 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee94698-787a-4bf1-a79b-d1ba4b24f656 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:30.860216 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee94698-787a-4bf1-a79b-d1ba4b24f656 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:30.864772 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f58626-04f0-4540-a586-8a680ff23a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:30.865363 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f58626-04f0-4540-a586-8a680ff23a7e admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:30.865644 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f58626-04f0-4540-a586-8a680ff23a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:30.865954 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f58626-04f0-4540-a586-8a680ff23a7e admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:30.867093 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f58626-04f0-4540-a586-8a680ff23a7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:30.868682 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f58626-04f0-4540-a586-8a680ff23a7e admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:30.878382 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1256/2526] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:24:30 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 09:24:30.882945 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3a3b72-b2f4-4b43-8ef7-569bcb07775d admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:30.892830 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3a3b72-b2f4-4b43-8ef7-569bcb07775d admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:30.897313 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f55c5a0-d8b8-483a-9b8c-898bf168cc3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:30.898012 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f55c5a0-d8b8-483a-9b8c-898bf168cc3e admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:30.898289 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f55c5a0-d8b8-483a-9b8c-898bf168cc3e admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:30.898563 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f55c5a0-d8b8-483a-9b8c-898bf168cc3e admin admin] PATH_INFO: `/v3/projects/55fe3f9b86724b5c8281d51d5dde6c68/users/e2134ea64ca54be7979ad2ff0b012193/roles/3248f0268738489b883041851d681bb7` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:30.912814 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f55c5a0-d8b8-483a-9b8c-898bf168cc3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=55fe3f9b86724b5c8281d51d5dde6c68, user_id=e2134ea64ca54be7979ad2ff0b012193, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:30.915458 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f55c5a0-d8b8-483a-9b8c-898bf168cc3e admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:30.928136 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1271/2527] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:24:30 2022] PUT /identity/v3/projects/55fe3f9b86724b5c8281d51d5dde6c68/users/e2134ea64ca54be7979ad2ff0b012193/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:24:30.938553 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0af20cd-bcbc-4d6a-9c83-2ea8dfef26e1 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:30.938778 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0af20cd-bcbc-4d6a-9c83-2ea8dfef26e1 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:30.939051 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0af20cd-bcbc-4d6a-9c83-2ea8dfef26e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:30.968325 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-a0af20cd-bcbc-4d6a-9c83-2ea8dfef26e1 None None] MFA Rules not processed for user `e2134ea64ca54be7979ad2ff0b012193`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:24:30.969595 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-a0af20cd-bcbc-4d6a-9c83-2ea8dfef26e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:24:31.031993 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1257/2528] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:24:30 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:24:31.036557 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f55c5a0-d8b8-483a-9b8c-898bf168cc3e admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:31.100616 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f55c5a0-d8b8-483a-9b8c-898bf168cc3e admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:31.105900 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb25a3c-121f-4ca6-a03d-35830307f849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:31.106564 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb25a3c-121f-4ca6-a03d-35830307f849 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:31.106856 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb25a3c-121f-4ca6-a03d-35830307f849 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:31.107171 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb25a3c-121f-4ca6-a03d-35830307f849 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:31.107805 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb25a3c-121f-4ca6-a03d-35830307f849 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:31.109748 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb25a3c-121f-4ca6-a03d-35830307f849 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:31.140238 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1272/2529] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 09:24:31 2022] POST /identity/v3/projects => generated 377 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:24:31.141417 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0af20cd-bcbc-4d6a-9c83-2ea8dfef26e1 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:31.152454 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0af20cd-bcbc-4d6a-9c83-2ea8dfef26e1 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:31.210604 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02567c65-0f1b-4eb8-8aa7-e59c6d363567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:31.211122 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02567c65-0f1b-4eb8-8aa7-e59c6d363567 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:31.211402 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02567c65-0f1b-4eb8-8aa7-e59c6d363567 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:31.211674 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02567c65-0f1b-4eb8-8aa7-e59c6d363567 admin admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:31.212292 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02567c65-0f1b-4eb8-8aa7-e59c6d363567 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:31.214229 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02567c65-0f1b-4eb8-8aa7-e59c6d363567 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:31.235494 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1258/2530] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 09:24:31 2022] POST /identity/v3/users => generated 295 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:24:31.239499 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb25a3c-121f-4ca6-a03d-35830307f849 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:31.250180 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb25a3c-121f-4ca6-a03d-35830307f849 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:31.254704 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778f39db-f24e-44aa-b0f8-ef2a469ad65b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:31.255292 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778f39db-f24e-44aa-b0f8-ef2a469ad65b admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:31.255572 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778f39db-f24e-44aa-b0f8-ef2a469ad65b admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:31.255848 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778f39db-f24e-44aa-b0f8-ef2a469ad65b admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:31.256478 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778f39db-f24e-44aa-b0f8-ef2a469ad65b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:31.258112 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778f39db-f24e-44aa-b0f8-ef2a469ad65b admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:31.268967 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1273/2531] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:24:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:24:31.272831 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02567c65-0f1b-4eb8-8aa7-e59c6d363567 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:31.282902 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02567c65-0f1b-4eb8-8aa7-e59c6d363567 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:31.287463 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f05a81f-14c6-48ac-95a1-21cf19832daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:31.288106 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f05a81f-14c6-48ac-95a1-21cf19832daf admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:31.288939 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f05a81f-14c6-48ac-95a1-21cf19832daf admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:31.289236 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f05a81f-14c6-48ac-95a1-21cf19832daf admin admin] PATH_INFO: `/v3/projects/22d322bd493d4f418c7fa201d9ec7db1/users/4b75f7dbd96d4c67923021d0e03a44fb/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:31.303936 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f05a81f-14c6-48ac-95a1-21cf19832daf admin admin] RBAC: Authorizing `identity:create_grant(project_id=22d322bd493d4f418c7fa201d9ec7db1, user_id=4b75f7dbd96d4c67923021d0e03a44fb, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:31.307226 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f05a81f-14c6-48ac-95a1-21cf19832daf admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:31.319500 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1259/2532] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:24:31 2022] PUT /identity/v3/projects/22d322bd493d4f418c7fa201d9ec7db1/users/4b75f7dbd96d4c67923021d0e03a44fb/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 09:24:31.323370 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778f39db-f24e-44aa-b0f8-ef2a469ad65b admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:31.333860 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778f39db-f24e-44aa-b0f8-ef2a469ad65b admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:31.359388 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:24:31.391292 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85418024-83db-4c5e-9bdc-d961ea02d27b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:31.391902 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85418024-83db-4c5e-9bdc-d961ea02d27b admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:31.392187 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85418024-83db-4c5e-9bdc-d961ea02d27b admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:31.392464 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85418024-83db-4c5e-9bdc-d961ea02d27b admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:31.393102 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85418024-83db-4c5e-9bdc-d961ea02d27b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:31.394734 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85418024-83db-4c5e-9bdc-d961ea02d27b admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:31.405242 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1274/2533] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:24:31 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:24:31.409524 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f05a81f-14c6-48ac-95a1-21cf19832daf admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:31.420402 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f05a81f-14c6-48ac-95a1-21cf19832daf admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:31.424994 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da897ae2-5c9c-411a-ba27-1b78412a605b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:31.425640 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da897ae2-5c9c-411a-ba27-1b78412a605b admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:31.425958 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da897ae2-5c9c-411a-ba27-1b78412a605b admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:31.426238 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da897ae2-5c9c-411a-ba27-1b78412a605b admin admin] PATH_INFO: `/v3/projects/22d322bd493d4f418c7fa201d9ec7db1/users/4b75f7dbd96d4c67923021d0e03a44fb/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:31.428441 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da897ae2-5c9c-411a-ba27-1b78412a605b admin admin] RBAC: Authorizing `identity:create_grant(project_id=22d322bd493d4f418c7fa201d9ec7db1, user_id=4b75f7dbd96d4c67923021d0e03a44fb, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:31.431121 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da897ae2-5c9c-411a-ba27-1b78412a605b admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:31.441111 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1260/2534] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:24:31 2022] PUT /identity/v3/projects/22d322bd493d4f418c7fa201d9ec7db1/users/4b75f7dbd96d4c67923021d0e03a44fb/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:24:31.451047 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e980ee4-3544-416e-9a7d-904588d92953 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:31.451341 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e980ee4-3544-416e-9a7d-904588d92953 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:31.451652 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e980ee4-3544-416e-9a7d-904588d92953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:31.480202 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-2e980ee4-3544-416e-9a7d-904588d92953 None None] MFA Rules not processed for user `4b75f7dbd96d4c67923021d0e03a44fb`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:24:31.481396 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-2e980ee4-3544-416e-9a7d-904588d92953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:24:31.547169 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1275/2535] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:24:31 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:24:31.558610 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da897ae2-5c9c-411a-ba27-1b78412a605b admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:31.560362 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-da897ae2-5c9c-411a-ba27-1b78412a605b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:24:31.582320 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:24:31.618846 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da897ae2-5c9c-411a-ba27-1b78412a605b admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:31.622448 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1503c24-f002-483f-bd68-ff4b46b0e177 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:31.623046 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1503c24-f002-483f-bd68-ff4b46b0e177 service cinder] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:31.623358 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1503c24-f002-483f-bd68-ff4b46b0e177 service cinder] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:31.623646 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1503c24-f002-483f-bd68-ff4b46b0e177 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:31.632441 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1503c24-f002-483f-bd68-ff4b46b0e177 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:31.633918 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1503c24-f002-483f-bd68-ff4b46b0e177 service cinder] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:31.643197 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1261/2536] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:24:31 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:24:31.685818 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e980ee4-3544-416e-9a7d-904588d92953 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:31.687195 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-2e980ee4-3544-416e-9a7d-904588d92953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:24:31.699252 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e980ee4-3544-416e-9a7d-904588d92953 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:31.702779 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1d96c1-b62a-464c-a432-c6f80d60b419 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:31.703372 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d96c1-b62a-464c-a432-c6f80d60b419 service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:31.703651 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d96c1-b62a-464c-a432-c6f80d60b419 service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:31.703922 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d96c1-b62a-464c-a432-c6f80d60b419 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:31.762756 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1d96c1-b62a-464c-a432-c6f80d60b419 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:31.764166 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1d96c1-b62a-464c-a432-c6f80d60b419 service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:31.773781 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1276/2537] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:24:31 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:24:34.637541 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1503c24-f002-483f-bd68-ff4b46b0e177 service cinder] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:34.701003 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1503c24-f002-483f-bd68-ff4b46b0e177 service cinder] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:34.707007 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a0cf13-c3bd-4650-9c43-5a3c86e013c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:34.707632 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0cf13-c3bd-4650-9c43-5a3c86e013c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:34.707913 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0cf13-c3bd-4650-9c43-5a3c86e013c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:34.708182 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0cf13-c3bd-4650-9c43-5a3c86e013c7 admin admin] PATH_INFO: `/v3/users/d0d4611b152a4a1c9057de2b37a10dd9` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:34.709290 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a0cf13-c3bd-4650-9c43-5a3c86e013c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0d4611b152a4a1c9057de2b37a10dd9)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:34.711234 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a0cf13-c3bd-4650-9c43-5a3c86e013c7 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:34.758684 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-c9a0cf13-c3bd-4650-9c43-5a3c86e013c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0d4611b152a4a1c9057de2b37a10dd9'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:34.759339 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-c9a0cf13-c3bd-4650-9c43-5a3c86e013c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0d4611b152a4a1c9057de2b37a10dd9'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:34.771160 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-c9a0cf13-c3bd-4650-9c43-5a3c86e013c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0d4611b152a4a1c9057de2b37a10dd9'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:34.778202 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-c9a0cf13-c3bd-4650-9c43-5a3c86e013c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0d4611b152a4a1c9057de2b37a10dd9'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:34.778820 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-c9a0cf13-c3bd-4650-9c43-5a3c86e013c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0d4611b152a4a1c9057de2b37a10dd9'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:34.791362 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1262/2538] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 09:24:34 2022] DELETE /identity/v3/users/d0d4611b152a4a1c9057de2b37a10dd9 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:24:34.795783 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d32ab-1821-4d64-80ce-75e8d066b90e None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:34.796246 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d32ab-1821-4d64-80ce-75e8d066b90e None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:34.796339 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d32ab-1821-4d64-80ce-75e8d066b90e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:34.815430 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-852d32ab-1821-4d64-80ce-75e8d066b90e None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:24:34.816629 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-852d32ab-1821-4d64-80ce-75e8d066b90e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:24:34.859720 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:24:34.875240 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1277/2539] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:24:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:24:34.884986 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a0cf13-c3bd-4650-9c43-5a3c86e013c7 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:34.886609 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-c9a0cf13-c3bd-4650-9c43-5a3c86e013c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:24:34.931098 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:24:34.964256 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a0cf13-c3bd-4650-9c43-5a3c86e013c7 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:34.967804 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04950ccd-1903-4946-b794-e7bc08d5d02a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:34.968407 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04950ccd-1903-4946-b794-e7bc08d5d02a service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:34.968691 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04950ccd-1903-4946-b794-e7bc08d5d02a service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:34.968971 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04950ccd-1903-4946-b794-e7bc08d5d02a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:34.978088 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04950ccd-1903-4946-b794-e7bc08d5d02a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:34.980052 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04950ccd-1903-4946-b794-e7bc08d5d02a service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:34.989478 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1263/2540] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:24:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:24:35.252797 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852d32ab-1821-4d64-80ce-75e8d066b90e None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:35.254345 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-852d32ab-1821-4d64-80ce-75e8d066b90e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:24:35.270776 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852d32ab-1821-4d64-80ce-75e8d066b90e None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:35.277241 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefb2489-46ff-4863-aa61-5afb14c63b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:35.277835 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb2489-46ff-4863-aa61-5afb14c63b02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:35.278118 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb2489-46ff-4863-aa61-5afb14c63b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:35.278391 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb2489-46ff-4863-aa61-5afb14c63b02 admin admin] PATH_INFO: `/v3/projects/a7741fcacd2f41e6b4517552e233e12d` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:35.279642 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefb2489-46ff-4863-aa61-5afb14c63b02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7741fcacd2f41e6b4517552e233e12d)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:35.281522 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefb2489-46ff-4863-aa61-5afb14c63b02 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:35.328139 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-aefb2489-46ff-4863-aa61-5afb14c63b02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7741fcacd2f41e6b4517552e233e12d', 'request_id': 'req-aefb2489-46ff-4863-aa61-5afb14c63b02'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:35.334841 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-aefb2489-46ff-4863-aa61-5afb14c63b02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7741fcacd2f41e6b4517552e233e12d', 'request_id': 'req-aefb2489-46ff-4863-aa61-5afb14c63b02'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:35.341937 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-aefb2489-46ff-4863-aa61-5afb14c63b02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7741fcacd2f41e6b4517552e233e12d', 'request_id': 'req-aefb2489-46ff-4863-aa61-5afb14c63b02'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:35.352016 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-aefb2489-46ff-4863-aa61-5afb14c63b02 admin admin] The token cache is being invalidate because project a7741fcacd2f41e6b4517552e233e12d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 09:24:35.352306 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-aefb2489-46ff-4863-aa61-5afb14c63b02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:35.352917 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-aefb2489-46ff-4863-aa61-5afb14c63b02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:35.355046 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1278/2541] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 09:24:35 2022] DELETE /identity/v3/projects/a7741fcacd2f41e6b4517552e233e12d => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:24:35.359042 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04950ccd-1903-4946-b794-e7bc08d5d02a service neutron] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:35.368461 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04950ccd-1903-4946-b794-e7bc08d5d02a service neutron] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:35.424304 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35856b2-16af-4baf-863f-c3ff13531c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:35.424930 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35856b2-16af-4baf-863f-c3ff13531c27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:35.425209 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35856b2-16af-4baf-863f-c3ff13531c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:35.425485 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35856b2-16af-4baf-863f-c3ff13531c27 admin admin] PATH_INFO: `/v3/users/e2134ea64ca54be7979ad2ff0b012193` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:35.426622 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35856b2-16af-4baf-863f-c3ff13531c27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2134ea64ca54be7979ad2ff0b012193)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:35.428790 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35856b2-16af-4baf-863f-c3ff13531c27 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:35.473699 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-d35856b2-16af-4baf-863f-c3ff13531c27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2134ea64ca54be7979ad2ff0b012193'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:35.474344 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-d35856b2-16af-4baf-863f-c3ff13531c27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2134ea64ca54be7979ad2ff0b012193'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:35.486056 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-d35856b2-16af-4baf-863f-c3ff13531c27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2134ea64ca54be7979ad2ff0b012193'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:35.493608 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-d35856b2-16af-4baf-863f-c3ff13531c27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2134ea64ca54be7979ad2ff0b012193'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:35.494337 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-d35856b2-16af-4baf-863f-c3ff13531c27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2134ea64ca54be7979ad2ff0b012193'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:35.506183 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1264/2542] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 09:24:35 2022] DELETE /identity/v3/users/e2134ea64ca54be7979ad2ff0b012193 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:24:35.770292 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefb2489-46ff-4863-aa61-5afb14c63b02 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:35.771942 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-aefb2489-46ff-4863-aa61-5afb14c63b02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:24:35.789533 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefb2489-46ff-4863-aa61-5afb14c63b02 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:35.847891 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceda49f-7951-41ba-ad28-9c82ef18b1c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:35.848533 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceda49f-7951-41ba-ad28-9c82ef18b1c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:35.848800 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceda49f-7951-41ba-ad28-9c82ef18b1c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:35.849058 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceda49f-7951-41ba-ad28-9c82ef18b1c7 admin admin] PATH_INFO: `/v3/projects/55fe3f9b86724b5c8281d51d5dde6c68` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:35.850200 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceda49f-7951-41ba-ad28-9c82ef18b1c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55fe3f9b86724b5c8281d51d5dde6c68)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:35.852081 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceda49f-7951-41ba-ad28-9c82ef18b1c7 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:35.896854 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8ceda49f-7951-41ba-ad28-9c82ef18b1c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55fe3f9b86724b5c8281d51d5dde6c68', 'request_id': 'req-8ceda49f-7951-41ba-ad28-9c82ef18b1c7'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:35.903074 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8ceda49f-7951-41ba-ad28-9c82ef18b1c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55fe3f9b86724b5c8281d51d5dde6c68', 'request_id': 'req-8ceda49f-7951-41ba-ad28-9c82ef18b1c7'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:35.910066 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8ceda49f-7951-41ba-ad28-9c82ef18b1c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55fe3f9b86724b5c8281d51d5dde6c68', 'request_id': 'req-8ceda49f-7951-41ba-ad28-9c82ef18b1c7'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:35.920027 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8ceda49f-7951-41ba-ad28-9c82ef18b1c7 admin admin] The token cache is being invalidate because project 55fe3f9b86724b5c8281d51d5dde6c68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 09:24:35.920314 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8ceda49f-7951-41ba-ad28-9c82ef18b1c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:35.920940 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8ceda49f-7951-41ba-ad28-9c82ef18b1c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:35.922438 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1279/2543] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 09:24:35 2022] DELETE /identity/v3/projects/55fe3f9b86724b5c8281d51d5dde6c68 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:24:35.926426 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35856b2-16af-4baf-863f-c3ff13531c27 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:35.928114 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-d35856b2-16af-4baf-863f-c3ff13531c27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:24:35.940962 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35856b2-16af-4baf-863f-c3ff13531c27 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:35.981793 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:24:35.998638 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd99084-a449-4116-abba-e10a2f7575b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:35.999264 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd99084-a449-4116-abba-e10a2f7575b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:35.999541 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd99084-a449-4116-abba-e10a2f7575b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:35.999815 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd99084-a449-4116-abba-e10a2f7575b8 admin admin] PATH_INFO: `/v3/users/4b75f7dbd96d4c67923021d0e03a44fb` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:36.000904 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd99084-a449-4116-abba-e10a2f7575b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b75f7dbd96d4c67923021d0e03a44fb)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:36.002876 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd99084-a449-4116-abba-e10a2f7575b8 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:36.048484 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-3bd99084-a449-4116-abba-e10a2f7575b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b75f7dbd96d4c67923021d0e03a44fb'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:36.049058 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-3bd99084-a449-4116-abba-e10a2f7575b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b75f7dbd96d4c67923021d0e03a44fb'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:36.061682 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-3bd99084-a449-4116-abba-e10a2f7575b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b75f7dbd96d4c67923021d0e03a44fb'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:36.069135 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-3bd99084-a449-4116-abba-e10a2f7575b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b75f7dbd96d4c67923021d0e03a44fb'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:36.069779 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-3bd99084-a449-4116-abba-e10a2f7575b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b75f7dbd96d4c67923021d0e03a44fb'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:36.082315 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1265/2544] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 09:24:35 2022] DELETE /identity/v3/users/4b75f7dbd96d4c67923021d0e03a44fb => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:24:36.345515 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceda49f-7951-41ba-ad28-9c82ef18b1c7 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:36.346993 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-8ceda49f-7951-41ba-ad28-9c82ef18b1c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:24:36.354799 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:24:36.361853 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceda49f-7951-41ba-ad28-9c82ef18b1c7 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:36.421530 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c36612-08f4-42b1-a28e-3267a7e44f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:36.422324 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c36612-08f4-42b1-a28e-3267a7e44f8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:36.422603 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c36612-08f4-42b1-a28e-3267a7e44f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:36.422877 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c36612-08f4-42b1-a28e-3267a7e44f8a admin admin] PATH_INFO: `/v3/projects/22d322bd493d4f418c7fa201d9ec7db1` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:36.423973 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c36612-08f4-42b1-a28e-3267a7e44f8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=22d322bd493d4f418c7fa201d9ec7db1)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:36.425918 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c36612-08f4-42b1-a28e-3267a7e44f8a admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:36.472609 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b9c36612-08f4-42b1-a28e-3267a7e44f8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22d322bd493d4f418c7fa201d9ec7db1', 'request_id': 'req-b9c36612-08f4-42b1-a28e-3267a7e44f8a'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:36.478831 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b9c36612-08f4-42b1-a28e-3267a7e44f8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22d322bd493d4f418c7fa201d9ec7db1', 'request_id': 'req-b9c36612-08f4-42b1-a28e-3267a7e44f8a'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:36.485885 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b9c36612-08f4-42b1-a28e-3267a7e44f8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22d322bd493d4f418c7fa201d9ec7db1', 'request_id': 'req-b9c36612-08f4-42b1-a28e-3267a7e44f8a'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:36.496109 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b9c36612-08f4-42b1-a28e-3267a7e44f8a admin admin] The token cache is being invalidate because project 22d322bd493d4f418c7fa201d9ec7db1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 09:24:36.496393 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b9c36612-08f4-42b1-a28e-3267a7e44f8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:36.497049 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b9c36612-08f4-42b1-a28e-3267a7e44f8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:36.498544 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1280/2545] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 09:24:36 2022] DELETE /identity/v3/projects/22d322bd493d4f418c7fa201d9ec7db1 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:24:36.504494 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b15ed6-e8e7-4163-92b8-b7d778440728 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:36.504777 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b15ed6-e8e7-4163-92b8-b7d778440728 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:36.505050 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b15ed6-e8e7-4163-92b8-b7d778440728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:36.524813 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-c4b15ed6-e8e7-4163-92b8-b7d778440728 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:24:36.526655 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-c4b15ed6-e8e7-4163-92b8-b7d778440728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:24:36.583141 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1266/2546] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:24:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:24:36.592750 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3558ac88-6782-4f96-bb1a-43873ea542b7 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:36.592975 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3558ac88-6782-4f96-bb1a-43873ea542b7 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:36.593286 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3558ac88-6782-4f96-bb1a-43873ea542b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:36.612726 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-3558ac88-6782-4f96-bb1a-43873ea542b7 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:24:36.613969 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-3558ac88-6782-4f96-bb1a-43873ea542b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:24:36.622365 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1281/2547] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:24:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:24:36.626493 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b15ed6-e8e7-4163-92b8-b7d778440728 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:36.636649 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b15ed6-e8e7-4163-92b8-b7d778440728 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:36.641217 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f2c454-eaee-45ed-bcee-ed99848218d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:36.641854 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f2c454-eaee-45ed-bcee-ed99848218d3 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:36.642138 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f2c454-eaee-45ed-bcee-ed99848218d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:36.642413 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f2c454-eaee-45ed-bcee-ed99848218d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:36.643053 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f2c454-eaee-45ed-bcee-ed99848218d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 09:24:36.643384 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f2c454-eaee-45ed-bcee-ed99848218d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:36.644982 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f2c454-eaee-45ed-bcee-ed99848218d3 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:36.656294 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1267/2548] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 09:24:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 09:24:36.660566 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3558ac88-6782-4f96-bb1a-43873ea542b7 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:36.669747 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3558ac88-6782-4f96-bb1a-43873ea542b7 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:36.674410 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f14d41-e36a-4b0c-a3d2-93b920111c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:36.674995 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f14d41-e36a-4b0c-a3d2-93b920111c12 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:36.675275 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f14d41-e36a-4b0c-a3d2-93b920111c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:36.675549 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f14d41-e36a-4b0c-a3d2-93b920111c12 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:36.676173 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f14d41-e36a-4b0c-a3d2-93b920111c12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:36.678365 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f14d41-e36a-4b0c-a3d2-93b920111c12 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:36.705755 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1282/2549] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 09:24:36 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:24:36.709803 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f2c454-eaee-45ed-bcee-ed99848218d3 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:36.719959 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f2c454-eaee-45ed-bcee-ed99848218d3 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:36.766444 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:24:36.777789 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac63adb-e928-460f-8ea6-8dc3ac45292c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:36.778392 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac63adb-e928-460f-8ea6-8dc3ac45292c admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:36.778674 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac63adb-e928-460f-8ea6-8dc3ac45292c admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:36.778979 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac63adb-e928-460f-8ea6-8dc3ac45292c admin admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:36.779643 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac63adb-e928-460f-8ea6-8dc3ac45292c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:36.781607 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac63adb-e928-460f-8ea6-8dc3ac45292c admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:36.801607 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1268/2550] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 09:24:36 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:24:36.806149 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f14d41-e36a-4b0c-a3d2-93b920111c12 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:36.816760 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f14d41-e36a-4b0c-a3d2-93b920111c12 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:36.821546 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20654394-9c6f-4faf-a54e-691703a4b8fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:36.822158 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20654394-9c6f-4faf-a54e-691703a4b8fc admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:36.823022 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20654394-9c6f-4faf-a54e-691703a4b8fc admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:36.823394 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20654394-9c6f-4faf-a54e-691703a4b8fc admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:36.824030 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20654394-9c6f-4faf-a54e-691703a4b8fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:36.825601 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20654394-9c6f-4faf-a54e-691703a4b8fc admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:36.837264 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1283/2551] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:24:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:24:36.841217 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac63adb-e928-460f-8ea6-8dc3ac45292c admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:36.851090 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac63adb-e928-460f-8ea6-8dc3ac45292c admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:36.855609 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b384c3-99c1-4674-938a-666d82cda3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:36.856199 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b384c3-99c1-4674-938a-666d82cda3aa admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:36.856480 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b384c3-99c1-4674-938a-666d82cda3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:36.856755 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b384c3-99c1-4674-938a-666d82cda3aa admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:36.857389 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b384c3-99c1-4674-938a-666d82cda3aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:36.859026 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b384c3-99c1-4674-938a-666d82cda3aa admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:36.869960 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1269/2552] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:24:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 09:24:36.873797 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20654394-9c6f-4faf-a54e-691703a4b8fc admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:36.883532 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20654394-9c6f-4faf-a54e-691703a4b8fc admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:36.889021 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4979cd00-f73d-4449-a147-6436a7c5f26f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:36.889660 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4979cd00-f73d-4449-a147-6436a7c5f26f admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:36.890039 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4979cd00-f73d-4449-a147-6436a7c5f26f admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:36.890344 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4979cd00-f73d-4449-a147-6436a7c5f26f admin admin] PATH_INFO: `/v3/projects/7d9566db344f4a9ab14b1c3cb2dce689/users/5691e2fd3c5c4c2cafb338b69f54fd9e/roles/3248f0268738489b883041851d681bb7` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:36.904619 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4979cd00-f73d-4449-a147-6436a7c5f26f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d9566db344f4a9ab14b1c3cb2dce689, user_id=5691e2fd3c5c4c2cafb338b69f54fd9e, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:36.907316 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4979cd00-f73d-4449-a147-6436a7c5f26f admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:36.920313 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1284/2553] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:24:36 2022] PUT /identity/v3/projects/7d9566db344f4a9ab14b1c3cb2dce689/users/5691e2fd3c5c4c2cafb338b69f54fd9e/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 09:24:36.926025 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d390a-0dcb-426f-8437-6529a2ccefc8 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:36.926322 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d390a-0dcb-426f-8437-6529a2ccefc8 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:36.926597 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d390a-0dcb-426f-8437-6529a2ccefc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:36.947550 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-566d390a-0dcb-426f-8437-6529a2ccefc8 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:24:36.948746 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-566d390a-0dcb-426f-8437-6529a2ccefc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:24:37.005894 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1270/2554] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:24:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 09:24:37.015465 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4979cd00-f73d-4449-a147-6436a7c5f26f admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:37.017175 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-4979cd00-f73d-4449-a147-6436a7c5f26f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:24:37.048699 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:24:37.075735 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4979cd00-f73d-4449-a147-6436a7c5f26f admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:37.079364 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88e95a5-4540-4e85-822e-bd7fc9e184b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:37.079964 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e95a5-4540-4e85-822e-bd7fc9e184b2 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:37.080249 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e95a5-4540-4e85-822e-bd7fc9e184b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:37.080525 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e95a5-4540-4e85-822e-bd7fc9e184b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:37.089790 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88e95a5-4540-4e85-822e-bd7fc9e184b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:37.091208 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88e95a5-4540-4e85-822e-bd7fc9e184b2 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:37.104751 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1285/2555] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:24:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 09:24:37.715315 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f519aaa0-c50f-4fb9-98b5-5f7ae20c0cfe None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:37.715611 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f519aaa0-c50f-4fb9-98b5-5f7ae20c0cfe None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:37.715892 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f519aaa0-c50f-4fb9-98b5-5f7ae20c0cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:37.743887 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-f519aaa0-c50f-4fb9-98b5-5f7ae20c0cfe None None] MFA Rules not processed for user `5691e2fd3c5c4c2cafb338b69f54fd9e`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:24:37.745128 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-f519aaa0-c50f-4fb9-98b5-5f7ae20c0cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:24:37.814669 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1271/2556] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:24:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:24:37.819088 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88e95a5-4540-4e85-822e-bd7fc9e184b2 service neutron] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:37.829651 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88e95a5-4540-4e85-822e-bd7fc9e184b2 service neutron] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:37.834426 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87eb527-d24e-4cb7-a3cb-26147b9acf32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:37.835026 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87eb527-d24e-4cb7-a3cb-26147b9acf32 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:37.835305 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87eb527-d24e-4cb7-a3cb-26147b9acf32 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:37.835577 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87eb527-d24e-4cb7-a3cb-26147b9acf32 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:37.836244 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87eb527-d24e-4cb7-a3cb-26147b9acf32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:37.838768 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87eb527-d24e-4cb7-a3cb-26147b9acf32 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:37.868083 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1286/2557] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 09:24:37 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 09:24:37.869570 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f519aaa0-c50f-4fb9-98b5-5f7ae20c0cfe None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:37.880562 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f519aaa0-c50f-4fb9-98b5-5f7ae20c0cfe None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:37.938104 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7908b7-e91d-40e6-add9-84d6e670eff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:37.938730 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7908b7-e91d-40e6-add9-84d6e670eff2 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:37.938897 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7908b7-e91d-40e6-add9-84d6e670eff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:37.939165 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7908b7-e91d-40e6-add9-84d6e670eff2 admin admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:37.939781 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7908b7-e91d-40e6-add9-84d6e670eff2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:37.941701 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7908b7-e91d-40e6-add9-84d6e670eff2 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:37.949487 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:24:37.963017 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1272/2558] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 09:24:37 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:24:37.966980 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87eb527-d24e-4cb7-a3cb-26147b9acf32 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:37.977133 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87eb527-d24e-4cb7-a3cb-26147b9acf32 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:37.982294 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1894935-d47b-4dc3-9350-fc860bb70d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:37.982887 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1894935-d47b-4dc3-9350-fc860bb70d69 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:37.983171 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1894935-d47b-4dc3-9350-fc860bb70d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:37.983443 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1894935-d47b-4dc3-9350-fc860bb70d69 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:37.984073 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1894935-d47b-4dc3-9350-fc860bb70d69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:37.985649 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1894935-d47b-4dc3-9350-fc860bb70d69 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:37.997470 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1287/2559] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:24:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:24:38.001451 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7908b7-e91d-40e6-add9-84d6e670eff2 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:38.011109 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7908b7-e91d-40e6-add9-84d6e670eff2 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:38.015586 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3abd07-c7b5-4cd8-ac77-285c20a5f8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:38.016220 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3abd07-c7b5-4cd8-ac77-285c20a5f8be admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:38.016499 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3abd07-c7b5-4cd8-ac77-285c20a5f8be admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:38.016774 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3abd07-c7b5-4cd8-ac77-285c20a5f8be admin admin] PATH_INFO: `/v3/projects/5c9c35bbbcaf448fad5792477164f89d/users/f8596f54883c478f8ce5298c4d5382ea/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:38.031288 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3abd07-c7b5-4cd8-ac77-285c20a5f8be admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c9c35bbbcaf448fad5792477164f89d, user_id=f8596f54883c478f8ce5298c4d5382ea, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:38.034005 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3abd07-c7b5-4cd8-ac77-285c20a5f8be admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:38.047622 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1273/2560] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:24:38 2022] PUT /identity/v3/projects/5c9c35bbbcaf448fad5792477164f89d/users/f8596f54883c478f8ce5298c4d5382ea/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:24:38.051503 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1894935-d47b-4dc3-9350-fc860bb70d69 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:38.062302 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1894935-d47b-4dc3-9350-fc860bb70d69 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:38.118919 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab42065-1945-4ca2-853d-71b52957b2d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:38.119574 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab42065-1945-4ca2-853d-71b52957b2d3 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:38.119850 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab42065-1945-4ca2-853d-71b52957b2d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:38.120122 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab42065-1945-4ca2-853d-71b52957b2d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:38.120744 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab42065-1945-4ca2-853d-71b52957b2d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:38.122361 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab42065-1945-4ca2-853d-71b52957b2d3 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:38.132103 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1288/2561] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:24:38 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:24:38.136199 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3abd07-c7b5-4cd8-ac77-285c20a5f8be admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:38.146305 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3abd07-c7b5-4cd8-ac77-285c20a5f8be admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:38.151322 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0819861d-7e79-4661-b461-30d1c45ea72f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:38.151966 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0819861d-7e79-4661-b461-30d1c45ea72f admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:38.152251 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0819861d-7e79-4661-b461-30d1c45ea72f admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:38.152527 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0819861d-7e79-4661-b461-30d1c45ea72f admin admin] PATH_INFO: `/v3/projects/5c9c35bbbcaf448fad5792477164f89d/users/f8596f54883c478f8ce5298c4d5382ea/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:38.154442 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0819861d-7e79-4661-b461-30d1c45ea72f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c9c35bbbcaf448fad5792477164f89d, user_id=f8596f54883c478f8ce5298c4d5382ea, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:38.157046 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0819861d-7e79-4661-b461-30d1c45ea72f admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:38.167622 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1274/2562] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:24:38 2022] PUT /identity/v3/projects/5c9c35bbbcaf448fad5792477164f89d/users/f8596f54883c478f8ce5298c4d5382ea/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:24:38.784774 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34772392-005a-4a69-bd76-19051815e5a1 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:38.785059 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34772392-005a-4a69-bd76-19051815e5a1 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:38.785335 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34772392-005a-4a69-bd76-19051815e5a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:38.813998 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-34772392-005a-4a69-bd76-19051815e5a1 None None] MFA Rules not processed for user `f8596f54883c478f8ce5298c4d5382ea`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:24:38.815212 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-34772392-005a-4a69-bd76-19051815e5a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:24:38.882972 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1289/2563] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:24:38 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:24:38.895046 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0819861d-7e79-4661-b461-30d1c45ea72f admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:38.896719 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-0819861d-7e79-4661-b461-30d1c45ea72f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:24:38.956309 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0819861d-7e79-4661-b461-30d1c45ea72f admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:38.959903 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b8873-432a-4793-843f-c6c1346bc33f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:38.960510 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b8873-432a-4793-843f-c6c1346bc33f service cinder] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:38.960792 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b8873-432a-4793-843f-c6c1346bc33f service cinder] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:38.961121 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b8873-432a-4793-843f-c6c1346bc33f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:39.016106 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298b8873-432a-4793-843f-c6c1346bc33f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:39.017533 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298b8873-432a-4793-843f-c6c1346bc33f service cinder] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:39.026390 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1275/2564] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:24:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:24:40.374535 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34772392-005a-4a69-bd76-19051815e5a1 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:40.375898 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-34772392-005a-4a69-bd76-19051815e5a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:24:40.387861 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34772392-005a-4a69-bd76-19051815e5a1 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:40.391509 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743eeada-45dc-4fd4-bdfe-2c648f8edc80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:40.392091 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743eeada-45dc-4fd4-bdfe-2c648f8edc80 service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:40.392370 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743eeada-45dc-4fd4-bdfe-2c648f8edc80 service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:40.392668 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743eeada-45dc-4fd4-bdfe-2c648f8edc80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:40.402117 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743eeada-45dc-4fd4-bdfe-2c648f8edc80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:40.403538 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743eeada-45dc-4fd4-bdfe-2c648f8edc80 service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:40.412809 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1290/2565] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:24:40 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:24:49.022359 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:INSERT=5,keystone:SELECT=65 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:24:50.408633 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=96,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:24:52.744621 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b8873-432a-4793-843f-c6c1346bc33f service cinder] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:52.746110 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-298b8873-432a-4793-843f-c6c1346bc33f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:24:52.821007 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b8873-432a-4793-843f-c6c1346bc33f service cinder] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:52.826610 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136031c0-99fb-4114-90ab-c95503d91f8c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38986ca2612c4fc59d960cd693834f9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:52.827242 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136031c0-99fb-4114-90ab-c95503d91f8c service placement] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:52.827504 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136031c0-99fb-4114-90ab-c95503d91f8c service placement] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:52.827779 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136031c0-99fb-4114-90ab-c95503d91f8c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:52.829586 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-136031c0-99fb-4114-90ab-c95503d91f8c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:24:52.841766 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136031c0-99fb-4114-90ab-c95503d91f8c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:52.843347 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136031c0-99fb-4114-90ab-c95503d91f8c service placement] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:52.853814 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1276/2566] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:24:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:24:53.239933 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743eeada-45dc-4fd4-bdfe-2c648f8edc80 service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:53.252369 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743eeada-45dc-4fd4-bdfe-2c648f8edc80 service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:53.257354 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b477af8a-4241-4774-897c-305b4d302913 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38986ca2612c4fc59d960cd693834f9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:53.257992 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b477af8a-4241-4774-897c-305b4d302913 service placement] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:53.258274 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b477af8a-4241-4774-897c-305b4d302913 service placement] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:53.258546 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b477af8a-4241-4774-897c-305b4d302913 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:53.260834 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-b477af8a-4241-4774-897c-305b4d302913 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:24:53.272252 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b477af8a-4241-4774-897c-305b4d302913 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:53.273850 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b477af8a-4241-4774-897c-305b4d302913 service placement] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:53.284981 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1291/2567] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:24:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:24:53.295503 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136031c0-99fb-4114-90ab-c95503d91f8c service placement] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:53.363103 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136031c0-99fb-4114-90ab-c95503d91f8c service placement] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:53.367625 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951805d8-1d4b-4fc1-8787-1da59d88cb9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:53.368255 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951805d8-1d4b-4fc1-8787-1da59d88cb9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:53.368538 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951805d8-1d4b-4fc1-8787-1da59d88cb9d admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:53.368816 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951805d8-1d4b-4fc1-8787-1da59d88cb9d admin admin] PATH_INFO: `/v3/users/5691e2fd3c5c4c2cafb338b69f54fd9e` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:53.370498 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951805d8-1d4b-4fc1-8787-1da59d88cb9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5691e2fd3c5c4c2cafb338b69f54fd9e)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:53.372451 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951805d8-1d4b-4fc1-8787-1da59d88cb9d admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:53.418361 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-951805d8-1d4b-4fc1-8787-1da59d88cb9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5691e2fd3c5c4c2cafb338b69f54fd9e'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:53.419023 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-951805d8-1d4b-4fc1-8787-1da59d88cb9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5691e2fd3c5c4c2cafb338b69f54fd9e'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:53.431148 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-951805d8-1d4b-4fc1-8787-1da59d88cb9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5691e2fd3c5c4c2cafb338b69f54fd9e'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:53.438911 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-951805d8-1d4b-4fc1-8787-1da59d88cb9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5691e2fd3c5c4c2cafb338b69f54fd9e'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:53.439529 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-951805d8-1d4b-4fc1-8787-1da59d88cb9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5691e2fd3c5c4c2cafb338b69f54fd9e'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:53.451712 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1277/2568] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 09:24:53 2022] DELETE /identity/v3/users/5691e2fd3c5c4c2cafb338b69f54fd9e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 09:24:53.618932 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b477af8a-4241-4774-897c-305b4d302913 service placement] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:53.631717 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b477af8a-4241-4774-897c-305b4d302913 service placement] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:53.693859 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b12b76-7f18-4ba0-a44d-c2c7389f3e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:53.694410 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b12b76-7f18-4ba0-a44d-c2c7389f3e5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:53.694695 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b12b76-7f18-4ba0-a44d-c2c7389f3e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:53.694970 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b12b76-7f18-4ba0-a44d-c2c7389f3e5a admin admin] PATH_INFO: `/v3/projects/7d9566db344f4a9ab14b1c3cb2dce689` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:53.696073 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b12b76-7f18-4ba0-a44d-c2c7389f3e5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d9566db344f4a9ab14b1c3cb2dce689)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:53.698086 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b12b76-7f18-4ba0-a44d-c2c7389f3e5a admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:53.743361 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-e4b12b76-7f18-4ba0-a44d-c2c7389f3e5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d9566db344f4a9ab14b1c3cb2dce689', 'request_id': 'req-e4b12b76-7f18-4ba0-a44d-c2c7389f3e5a'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:53.750110 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-e4b12b76-7f18-4ba0-a44d-c2c7389f3e5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d9566db344f4a9ab14b1c3cb2dce689', 'request_id': 'req-e4b12b76-7f18-4ba0-a44d-c2c7389f3e5a'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:53.757443 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-e4b12b76-7f18-4ba0-a44d-c2c7389f3e5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d9566db344f4a9ab14b1c3cb2dce689', 'request_id': 'req-e4b12b76-7f18-4ba0-a44d-c2c7389f3e5a'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:53.767540 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-e4b12b76-7f18-4ba0-a44d-c2c7389f3e5a admin admin] The token cache is being invalidate because project 7d9566db344f4a9ab14b1c3cb2dce689 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 09:24:53.767828 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-e4b12b76-7f18-4ba0-a44d-c2c7389f3e5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:53.768521 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-e4b12b76-7f18-4ba0-a44d-c2c7389f3e5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:53.770076 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1292/2569] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 09:24:53 2022] DELETE /identity/v3/projects/7d9566db344f4a9ab14b1c3cb2dce689 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:24:53.774007 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951805d8-1d4b-4fc1-8787-1da59d88cb9d admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:53.775406 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-951805d8-1d4b-4fc1-8787-1da59d88cb9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:24:53.788703 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951805d8-1d4b-4fc1-8787-1da59d88cb9d admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:53.793812 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:24:53.845746 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d7b9ec-1b25-4a60-a43b-a0b3a9fcd5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:53.846372 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d7b9ec-1b25-4a60-a43b-a0b3a9fcd5e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:53.846654 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d7b9ec-1b25-4a60-a43b-a0b3a9fcd5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:53.846930 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d7b9ec-1b25-4a60-a43b-a0b3a9fcd5e1 admin admin] PATH_INFO: `/v3/users/f8596f54883c478f8ce5298c4d5382ea` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:53.848042 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d7b9ec-1b25-4a60-a43b-a0b3a9fcd5e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8596f54883c478f8ce5298c4d5382ea)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:53.850061 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d7b9ec-1b25-4a60-a43b-a0b3a9fcd5e1 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:53.894119 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-33d7b9ec-1b25-4a60-a43b-a0b3a9fcd5e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8596f54883c478f8ce5298c4d5382ea'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:53.894763 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-33d7b9ec-1b25-4a60-a43b-a0b3a9fcd5e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8596f54883c478f8ce5298c4d5382ea'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:53.907077 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-33d7b9ec-1b25-4a60-a43b-a0b3a9fcd5e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8596f54883c478f8ce5298c4d5382ea'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:53.913923 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-33d7b9ec-1b25-4a60-a43b-a0b3a9fcd5e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8596f54883c478f8ce5298c4d5382ea'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:53.914646 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-33d7b9ec-1b25-4a60-a43b-a0b3a9fcd5e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8596f54883c478f8ce5298c4d5382ea'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:53.927032 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1278/2570] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 09:24:53 2022] DELETE /identity/v3/users/f8596f54883c478f8ce5298c4d5382ea => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:24:54.095345 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b12b76-7f18-4ba0-a44d-c2c7389f3e5a admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:54.096804 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-e4b12b76-7f18-4ba0-a44d-c2c7389f3e5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:24:54.110771 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b12b76-7f18-4ba0-a44d-c2c7389f3e5a admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:54.170001 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5f4742-e2f1-4020-96ed-d3997c0a31c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:54.170656 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f4742-e2f1-4020-96ed-d3997c0a31c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:54.170936 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f4742-e2f1-4020-96ed-d3997c0a31c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:54.171209 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f4742-e2f1-4020-96ed-d3997c0a31c8 admin admin] PATH_INFO: `/v3/projects/5c9c35bbbcaf448fad5792477164f89d` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:54.172362 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5f4742-e2f1-4020-96ed-d3997c0a31c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c9c35bbbcaf448fad5792477164f89d)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:54.174305 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5f4742-e2f1-4020-96ed-d3997c0a31c8 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:54.194894 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:24:54.221377 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-ec5f4742-e2f1-4020-96ed-d3997c0a31c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c9c35bbbcaf448fad5792477164f89d', 'request_id': 'req-ec5f4742-e2f1-4020-96ed-d3997c0a31c8'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:54.227531 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-ec5f4742-e2f1-4020-96ed-d3997c0a31c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c9c35bbbcaf448fad5792477164f89d', 'request_id': 'req-ec5f4742-e2f1-4020-96ed-d3997c0a31c8'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:54.234641 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-ec5f4742-e2f1-4020-96ed-d3997c0a31c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c9c35bbbcaf448fad5792477164f89d', 'request_id': 'req-ec5f4742-e2f1-4020-96ed-d3997c0a31c8'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:54.244917 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-ec5f4742-e2f1-4020-96ed-d3997c0a31c8 admin admin] The token cache is being invalidate because project 5c9c35bbbcaf448fad5792477164f89d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 09:24:54.245523 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-ec5f4742-e2f1-4020-96ed-d3997c0a31c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:54.245878 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-ec5f4742-e2f1-4020-96ed-d3997c0a31c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:24:54.247438 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1293/2571] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 09:24:54 2022] DELETE /identity/v3/projects/5c9c35bbbcaf448fad5792477164f89d => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:24:54.252351 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7b11b9-f9f2-4840-a8ee-b9b28969bf6f None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:54.252641 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7b11b9-f9f2-4840-a8ee-b9b28969bf6f None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:54.252915 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7b11b9-f9f2-4840-a8ee-b9b28969bf6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:54.272737 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-4e7b11b9-f9f2-4840-a8ee-b9b28969bf6f None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:24:54.274006 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-4e7b11b9-f9f2-4840-a8ee-b9b28969bf6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:24:54.331901 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1279/2572] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:24:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:24:54.340255 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1a684e-8977-4b45-a360-f32a79027019 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:54.340546 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1a684e-8977-4b45-a360-f32a79027019 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:54.340825 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1a684e-8977-4b45-a360-f32a79027019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:54.360077 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-6f1a684e-8977-4b45-a360-f32a79027019 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:24:54.361283 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-6f1a684e-8977-4b45-a360-f32a79027019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:24:54.369055 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1294/2573] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:24:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:24:54.373191 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7b11b9-f9f2-4840-a8ee-b9b28969bf6f None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:54.382882 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7b11b9-f9f2-4840-a8ee-b9b28969bf6f None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:54.387994 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff84c65-2cde-4d5a-b90d-75e2b54e8430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:54.388591 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff84c65-2cde-4d5a-b90d-75e2b54e8430 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:54.388872 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff84c65-2cde-4d5a-b90d-75e2b54e8430 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:54.389147 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff84c65-2cde-4d5a-b90d-75e2b54e8430 admin admin] PATH_INFO: `/v3/domains` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:54.389815 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff84c65-2cde-4d5a-b90d-75e2b54e8430 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 09:24:54.390120 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff84c65-2cde-4d5a-b90d-75e2b54e8430 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:54.391714 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff84c65-2cde-4d5a-b90d-75e2b54e8430 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:54.404408 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1280/2574] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 09:24:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 09:24:54.408859 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1a684e-8977-4b45-a360-f32a79027019 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:54.418155 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1a684e-8977-4b45-a360-f32a79027019 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:54.422875 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44bc3f2-4bd2-45ea-b55d-bcfc6ad2470c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:54.423469 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44bc3f2-4bd2-45ea-b55d-bcfc6ad2470c admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:54.423754 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44bc3f2-4bd2-45ea-b55d-bcfc6ad2470c admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:54.424032 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44bc3f2-4bd2-45ea-b55d-bcfc6ad2470c admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:54.424672 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44bc3f2-4bd2-45ea-b55d-bcfc6ad2470c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:54.426602 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44bc3f2-4bd2-45ea-b55d-bcfc6ad2470c admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:54.453296 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1295/2575] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 09:24:54 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:24:54.457354 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff84c65-2cde-4d5a-b90d-75e2b54e8430 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:54.467921 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff84c65-2cde-4d5a-b90d-75e2b54e8430 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:54.479068 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:24:54.524661 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9744434-5001-4e66-be7d-250e162838fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:54.525268 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9744434-5001-4e66-be7d-250e162838fb admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:54.525551 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9744434-5001-4e66-be7d-250e162838fb admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:54.526480 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9744434-5001-4e66-be7d-250e162838fb admin admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:54.527107 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9744434-5001-4e66-be7d-250e162838fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:54.529023 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9744434-5001-4e66-be7d-250e162838fb admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:54.550532 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1281/2576] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 09:24:54 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:24:54.554931 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44bc3f2-4bd2-45ea-b55d-bcfc6ad2470c admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:54.565743 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44bc3f2-4bd2-45ea-b55d-bcfc6ad2470c admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:54.570367 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f0b9e8-968c-4fbd-9ba8-4933cfef3d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:54.570962 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f0b9e8-968c-4fbd-9ba8-4933cfef3d20 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:54.571248 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f0b9e8-968c-4fbd-9ba8-4933cfef3d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:54.571526 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f0b9e8-968c-4fbd-9ba8-4933cfef3d20 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:54.572167 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f0b9e8-968c-4fbd-9ba8-4933cfef3d20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:54.574401 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f0b9e8-968c-4fbd-9ba8-4933cfef3d20 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:54.584121 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1296/2577] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:24:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:24:54.587851 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9744434-5001-4e66-be7d-250e162838fb admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:54.597524 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9744434-5001-4e66-be7d-250e162838fb admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:54.602180 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f150021-103f-4966-8b82-c5d8bc32da38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:54.602771 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f150021-103f-4966-8b82-c5d8bc32da38 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:54.603058 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f150021-103f-4966-8b82-c5d8bc32da38 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:54.603362 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f150021-103f-4966-8b82-c5d8bc32da38 admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:54.603981 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f150021-103f-4966-8b82-c5d8bc32da38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:54.605573 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f150021-103f-4966-8b82-c5d8bc32da38 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:54.616164 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1282/2578] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:24:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:24:54.620641 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f0b9e8-968c-4fbd-9ba8-4933cfef3d20 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:54.630718 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f0b9e8-968c-4fbd-9ba8-4933cfef3d20 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:54.636033 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83462d53-6c1e-43d4-96d0-6c5d15a3f7a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:54.636682 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83462d53-6c1e-43d4-96d0-6c5d15a3f7a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:54.636966 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83462d53-6c1e-43d4-96d0-6c5d15a3f7a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:54.637242 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83462d53-6c1e-43d4-96d0-6c5d15a3f7a5 admin admin] PATH_INFO: `/v3/projects/c595772a09a547348ff9a885ccfa916d/users/6bfea7152c8e406d9a765b7e5e2fbbf7/roles/3248f0268738489b883041851d681bb7` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:54.651555 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83462d53-6c1e-43d4-96d0-6c5d15a3f7a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c595772a09a547348ff9a885ccfa916d, user_id=6bfea7152c8e406d9a765b7e5e2fbbf7, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:54.654220 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83462d53-6c1e-43d4-96d0-6c5d15a3f7a5 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:54.666580 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1297/2579] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:24:54 2022] PUT /identity/v3/projects/c595772a09a547348ff9a885ccfa916d/users/6bfea7152c8e406d9a765b7e5e2fbbf7/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:24:54.676875 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb23578-d030-4d2c-8d4d-bd4f579569d6 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:54.677168 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb23578-d030-4d2c-8d4d-bd4f579569d6 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:54.677449 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb23578-d030-4d2c-8d4d-bd4f579569d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:54.706334 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-4bb23578-d030-4d2c-8d4d-bd4f579569d6 None None] MFA Rules not processed for user `6bfea7152c8e406d9a765b7e5e2fbbf7`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:24:54.707553 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-4bb23578-d030-4d2c-8d4d-bd4f579569d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:24:54.773176 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1283/2580] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:24:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 09:24:54.783772 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83462d53-6c1e-43d4-96d0-6c5d15a3f7a5 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:24:54.785449 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-83462d53-6c1e-43d4-96d0-6c5d15a3f7a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:24:54.841909 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83462d53-6c1e-43d4-96d0-6c5d15a3f7a5 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:24:54.845471 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1bd144-fd0d-4e54-8623-013ce2de54be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:24:54.846102 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1bd144-fd0d-4e54-8623-013ce2de54be service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:24:54.846379 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1bd144-fd0d-4e54-8623-013ce2de54be service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:24:54.846652 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1bd144-fd0d-4e54-8623-013ce2de54be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:24:54.854682 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1bd144-fd0d-4e54-8623-013ce2de54be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:24:54.856102 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1bd144-fd0d-4e54-8623-013ce2de54be service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:24:54.864765 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1298/2581] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:24:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:25:03.253323 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb23578-d030-4d2c-8d4d-bd4f579569d6 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:03.314244 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb23578-d030-4d2c-8d4d-bd4f579569d6 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:03.318682 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444a2ea4-60b6-4087-8cd0-76c3accbdce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:03.319302 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444a2ea4-60b6-4087-8cd0-76c3accbdce9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:03.319583 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444a2ea4-60b6-4087-8cd0-76c3accbdce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:03.319858 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444a2ea4-60b6-4087-8cd0-76c3accbdce9 admin admin] PATH_INFO: `/v3/users/6bfea7152c8e406d9a765b7e5e2fbbf7` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:03.320966 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444a2ea4-60b6-4087-8cd0-76c3accbdce9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bfea7152c8e406d9a765b7e5e2fbbf7)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:03.323013 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444a2ea4-60b6-4087-8cd0-76c3accbdce9 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:03.350167 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:25:03.369578 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-444a2ea4-60b6-4087-8cd0-76c3accbdce9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bfea7152c8e406d9a765b7e5e2fbbf7'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:03.370254 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-444a2ea4-60b6-4087-8cd0-76c3accbdce9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bfea7152c8e406d9a765b7e5e2fbbf7'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:03.382330 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-444a2ea4-60b6-4087-8cd0-76c3accbdce9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bfea7152c8e406d9a765b7e5e2fbbf7'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:03.390059 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-444a2ea4-60b6-4087-8cd0-76c3accbdce9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bfea7152c8e406d9a765b7e5e2fbbf7'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:03.390671 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-444a2ea4-60b6-4087-8cd0-76c3accbdce9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bfea7152c8e406d9a765b7e5e2fbbf7'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:03.402965 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1284/2582] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 09:25:03 2022] DELETE /identity/v3/users/6bfea7152c8e406d9a765b7e5e2fbbf7 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:25:03.408035 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed36db0e-6196-4098-8116-d63d48a05f80 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:03.408507 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed36db0e-6196-4098-8116-d63d48a05f80 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:03.408614 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed36db0e-6196-4098-8116-d63d48a05f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:03.429190 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-ed36db0e-6196-4098-8116-d63d48a05f80 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:25:03.430415 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-ed36db0e-6196-4098-8116-d63d48a05f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:25:03.471073 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:25:03.490907 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1299/2583] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:25:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:25:03.500961 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444a2ea4-60b6-4087-8cd0-76c3accbdce9 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:03.502464 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-444a2ea4-60b6-4087-8cd0-76c3accbdce9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:25:03.558849 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444a2ea4-60b6-4087-8cd0-76c3accbdce9 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:03.563147 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f97398f-953c-4629-b07c-b5b4e3ed0c5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:03.563798 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f97398f-953c-4629-b07c-b5b4e3ed0c5a service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:03.564073 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f97398f-953c-4629-b07c-b5b4e3ed0c5a service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:03.564342 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f97398f-953c-4629-b07c-b5b4e3ed0c5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:03.573131 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f97398f-953c-4629-b07c-b5b4e3ed0c5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:03.574614 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f97398f-953c-4629-b07c-b5b4e3ed0c5a service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:03.584591 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1285/2584] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:25:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:25:03.846462 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed36db0e-6196-4098-8116-d63d48a05f80 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:03.847912 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-ed36db0e-6196-4098-8116-d63d48a05f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:25:03.860334 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed36db0e-6196-4098-8116-d63d48a05f80 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:03.865043 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e60f9e1-b9a8-4764-8427-4e134d01bac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:03.865695 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e60f9e1-b9a8-4764-8427-4e134d01bac1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:03.865998 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e60f9e1-b9a8-4764-8427-4e134d01bac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:03.866266 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e60f9e1-b9a8-4764-8427-4e134d01bac1 admin admin] PATH_INFO: `/v3/projects/c595772a09a547348ff9a885ccfa916d` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:03.867374 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e60f9e1-b9a8-4764-8427-4e134d01bac1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c595772a09a547348ff9a885ccfa916d)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:03.869310 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e60f9e1-b9a8-4764-8427-4e134d01bac1 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:03.916157 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8e60f9e1-b9a8-4764-8427-4e134d01bac1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c595772a09a547348ff9a885ccfa916d', 'request_id': 'req-8e60f9e1-b9a8-4764-8427-4e134d01bac1'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:03.922438 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8e60f9e1-b9a8-4764-8427-4e134d01bac1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c595772a09a547348ff9a885ccfa916d', 'request_id': 'req-8e60f9e1-b9a8-4764-8427-4e134d01bac1'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:03.930010 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8e60f9e1-b9a8-4764-8427-4e134d01bac1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c595772a09a547348ff9a885ccfa916d', 'request_id': 'req-8e60f9e1-b9a8-4764-8427-4e134d01bac1'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:03.940467 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8e60f9e1-b9a8-4764-8427-4e134d01bac1 admin admin] The token cache is being invalidate because project c595772a09a547348ff9a885ccfa916d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 09:25:03.940755 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8e60f9e1-b9a8-4764-8427-4e134d01bac1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:03.941387 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8e60f9e1-b9a8-4764-8427-4e134d01bac1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:03.942917 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1300/2585] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 09:25:03 2022] DELETE /identity/v3/projects/c595772a09a547348ff9a885ccfa916d => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:25:03.947797 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ac387d-d7ae-46c0-8513-03e5fba282b8 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:03.948085 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ac387d-d7ae-46c0-8513-03e5fba282b8 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:03.948362 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ac387d-d7ae-46c0-8513-03e5fba282b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:03.968623 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-71ac387d-d7ae-46c0-8513-03e5fba282b8 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:25:03.969861 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-71ac387d-d7ae-46c0-8513-03e5fba282b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:25:04.027876 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1286/2586] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:25:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:25:04.036019 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd167c-36fe-443d-9d3a-5ca8c93ea7c1 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:04.036306 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd167c-36fe-443d-9d3a-5ca8c93ea7c1 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:04.036579 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd167c-36fe-443d-9d3a-5ca8c93ea7c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:04.056828 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-4edd167c-36fe-443d-9d3a-5ca8c93ea7c1 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:25:04.058083 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-4edd167c-36fe-443d-9d3a-5ca8c93ea7c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:25:04.066629 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1301/2587] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:25:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:25:04.070741 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ac387d-d7ae-46c0-8513-03e5fba282b8 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:04.080618 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ac387d-d7ae-46c0-8513-03e5fba282b8 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:04.085291 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acd6bf7-fc7d-4413-a530-32c68526af33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:04.085920 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acd6bf7-fc7d-4413-a530-32c68526af33 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:04.086204 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acd6bf7-fc7d-4413-a530-32c68526af33 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:04.086483 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acd6bf7-fc7d-4413-a530-32c68526af33 admin admin] PATH_INFO: `/v3/domains` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:04.087160 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acd6bf7-fc7d-4413-a530-32c68526af33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 09:25:04.087465 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acd6bf7-fc7d-4413-a530-32c68526af33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:04.089069 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acd6bf7-fc7d-4413-a530-32c68526af33 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:04.100419 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1287/2588] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 09:25:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 09:25:04.104587 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd167c-36fe-443d-9d3a-5ca8c93ea7c1 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:04.114210 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd167c-36fe-443d-9d3a-5ca8c93ea7c1 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:04.118776 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd4c878-8f79-4df6-b1b2-9284ab57afe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:04.119354 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd4c878-8f79-4df6-b1b2-9284ab57afe0 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:04.119632 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd4c878-8f79-4df6-b1b2-9284ab57afe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:04.119929 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd4c878-8f79-4df6-b1b2-9284ab57afe0 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:04.120558 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd4c878-8f79-4df6-b1b2-9284ab57afe0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:04.122473 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd4c878-8f79-4df6-b1b2-9284ab57afe0 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:04.149851 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1302/2589] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 09:25:04 2022] POST /identity/v3/projects => generated 359 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:25:04.154087 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acd6bf7-fc7d-4413-a530-32c68526af33 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:04.164072 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acd6bf7-fc7d-4413-a530-32c68526af33 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:04.194773 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:25:04.222504 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d4905e-6a65-43fa-ad74-249b8471cd4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:04.223110 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d4905e-6a65-43fa-ad74-249b8471cd4b admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:04.223396 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d4905e-6a65-43fa-ad74-249b8471cd4b admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:04.223675 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d4905e-6a65-43fa-ad74-249b8471cd4b admin admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:04.224315 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d4905e-6a65-43fa-ad74-249b8471cd4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:04.226269 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d4905e-6a65-43fa-ad74-249b8471cd4b admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:04.247141 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1288/2590] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 09:25:04 2022] POST /identity/v3/users => generated 280 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:25:04.251263 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd4c878-8f79-4df6-b1b2-9284ab57afe0 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:04.261304 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd4c878-8f79-4df6-b1b2-9284ab57afe0 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:04.265853 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0633648f-dd28-4eb7-acf4-5ab9b7eb2abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:04.266442 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0633648f-dd28-4eb7-acf4-5ab9b7eb2abc admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:04.266722 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0633648f-dd28-4eb7-acf4-5ab9b7eb2abc admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:04.267000 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0633648f-dd28-4eb7-acf4-5ab9b7eb2abc admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:04.267967 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0633648f-dd28-4eb7-acf4-5ab9b7eb2abc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:04.269558 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0633648f-dd28-4eb7-acf4-5ab9b7eb2abc admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:04.279787 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1303/2591] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:25:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:25:04.283539 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d4905e-6a65-43fa-ad74-249b8471cd4b admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:04.293592 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d4905e-6a65-43fa-ad74-249b8471cd4b admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:04.298841 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df218dd9-35c3-4c1f-8f90-99a96d3bac3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:04.299435 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df218dd9-35c3-4c1f-8f90-99a96d3bac3d admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:04.299721 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df218dd9-35c3-4c1f-8f90-99a96d3bac3d admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:04.300000 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df218dd9-35c3-4c1f-8f90-99a96d3bac3d admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:04.300640 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df218dd9-35c3-4c1f-8f90-99a96d3bac3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:04.302271 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df218dd9-35c3-4c1f-8f90-99a96d3bac3d admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:04.313014 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1289/2592] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:25:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:25:04.316901 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0633648f-dd28-4eb7-acf4-5ab9b7eb2abc admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:04.326610 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0633648f-dd28-4eb7-acf4-5ab9b7eb2abc admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:04.331928 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1477fc67-e803-4380-a6f8-41466cf19fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:04.332604 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477fc67-e803-4380-a6f8-41466cf19fe9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:04.332891 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477fc67-e803-4380-a6f8-41466cf19fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:04.333169 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477fc67-e803-4380-a6f8-41466cf19fe9 admin admin] PATH_INFO: `/v3/projects/0aa2c695c43b41ebb8b05ffa370e0a8b/users/c63392f2fc9c4010bd44cc5cb9d1ccdb/roles/3248f0268738489b883041851d681bb7` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:04.348020 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1477fc67-e803-4380-a6f8-41466cf19fe9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0aa2c695c43b41ebb8b05ffa370e0a8b, user_id=c63392f2fc9c4010bd44cc5cb9d1ccdb, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:04.350683 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1477fc67-e803-4380-a6f8-41466cf19fe9 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:04.364281 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1304/2593] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:25:04 2022] PUT /identity/v3/projects/0aa2c695c43b41ebb8b05ffa370e0a8b/users/c63392f2fc9c4010bd44cc5cb9d1ccdb/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 09:25:04.374399 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b37c156-e61c-45f5-8494-87a272700536 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:04.374687 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b37c156-e61c-45f5-8494-87a272700536 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:04.374964 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b37c156-e61c-45f5-8494-87a272700536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:04.402310 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-8b37c156-e61c-45f5-8494-87a272700536 None None] MFA Rules not processed for user `c63392f2fc9c4010bd44cc5cb9d1ccdb`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:25:04.403525 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-8b37c156-e61c-45f5-8494-87a272700536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:25:04.467485 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1290/2594] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:25:04 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:25:04.504189 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1477fc67-e803-4380-a6f8-41466cf19fe9 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:04.505922 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-1477fc67-e803-4380-a6f8-41466cf19fe9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:25:04.562525 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1477fc67-e803-4380-a6f8-41466cf19fe9 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:04.566065 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af793ec7-0363-4420-ab75-ef292c32c8c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:04.566663 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af793ec7-0363-4420-ab75-ef292c32c8c8 service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:04.566945 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af793ec7-0363-4420-ab75-ef292c32c8c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:04.567222 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af793ec7-0363-4420-ab75-ef292c32c8c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:04.575348 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af793ec7-0363-4420-ab75-ef292c32c8c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:04.577397 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af793ec7-0363-4420-ab75-ef292c32c8c8 service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:04.586041 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1305/2595] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:25:04 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:25:04.601207 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b37c156-e61c-45f5-8494-87a272700536 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:04.662646 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b37c156-e61c-45f5-8494-87a272700536 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:04.667220 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b6c65e-3765-4140-b04e-7073337312c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:04.667846 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6c65e-3765-4140-b04e-7073337312c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:04.668130 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6c65e-3765-4140-b04e-7073337312c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:04.668409 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6c65e-3765-4140-b04e-7073337312c4 admin admin] PATH_INFO: `/v3/users/c63392f2fc9c4010bd44cc5cb9d1ccdb` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:04.670095 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b6c65e-3765-4140-b04e-7073337312c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c63392f2fc9c4010bd44cc5cb9d1ccdb)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:04.672024 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b6c65e-3765-4140-b04e-7073337312c4 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:04.718289 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:25:04.721074 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-f8b6c65e-3765-4140-b04e-7073337312c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c63392f2fc9c4010bd44cc5cb9d1ccdb'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:04.722160 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-f8b6c65e-3765-4140-b04e-7073337312c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c63392f2fc9c4010bd44cc5cb9d1ccdb'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:04.735392 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-f8b6c65e-3765-4140-b04e-7073337312c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c63392f2fc9c4010bd44cc5cb9d1ccdb'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:04.742253 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-f8b6c65e-3765-4140-b04e-7073337312c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c63392f2fc9c4010bd44cc5cb9d1ccdb'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:04.742886 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-f8b6c65e-3765-4140-b04e-7073337312c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c63392f2fc9c4010bd44cc5cb9d1ccdb'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:04.755080 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1291/2596] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 09:25:04 2022] DELETE /identity/v3/users/c63392f2fc9c4010bd44cc5cb9d1ccdb => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:25:04.759618 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd1cef5-47ab-4964-8703-96ae5c053216 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:04.759904 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd1cef5-47ab-4964-8703-96ae5c053216 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:04.760179 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd1cef5-47ab-4964-8703-96ae5c053216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:04.779665 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-2cd1cef5-47ab-4964-8703-96ae5c053216 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:25:04.780844 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-2cd1cef5-47ab-4964-8703-96ae5c053216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:25:04.784667 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:25:04.841176 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1306/2597] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:25:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:25:04.851222 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b6c65e-3765-4140-b04e-7073337312c4 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:04.852622 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-f8b6c65e-3765-4140-b04e-7073337312c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:25:04.911111 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b6c65e-3765-4140-b04e-7073337312c4 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:04.914827 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fcddc6-70ff-49d6-b02e-20868deacc85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:04.915438 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fcddc6-70ff-49d6-b02e-20868deacc85 service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:04.915717 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fcddc6-70ff-49d6-b02e-20868deacc85 service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:04.915991 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fcddc6-70ff-49d6-b02e-20868deacc85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:04.925730 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fcddc6-70ff-49d6-b02e-20868deacc85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:04.927164 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fcddc6-70ff-49d6-b02e-20868deacc85 service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:04.937391 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1292/2598] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:25:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:25:05.201617 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd1cef5-47ab-4964-8703-96ae5c053216 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:05.203093 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-2cd1cef5-47ab-4964-8703-96ae5c053216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:25:05.215022 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd1cef5-47ab-4964-8703-96ae5c053216 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:05.220555 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e79545-7416-4daa-ac13-d9a0226503e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:05.221193 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e79545-7416-4daa-ac13-d9a0226503e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:05.221476 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e79545-7416-4daa-ac13-d9a0226503e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:05.221781 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e79545-7416-4daa-ac13-d9a0226503e9 admin admin] PATH_INFO: `/v3/projects/0aa2c695c43b41ebb8b05ffa370e0a8b` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:05.222937 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e79545-7416-4daa-ac13-d9a0226503e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0aa2c695c43b41ebb8b05ffa370e0a8b)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:05.224843 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e79545-7416-4daa-ac13-d9a0226503e9 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:05.272413 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b8e79545-7416-4daa-ac13-d9a0226503e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0aa2c695c43b41ebb8b05ffa370e0a8b', 'request_id': 'req-b8e79545-7416-4daa-ac13-d9a0226503e9'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:05.278684 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b8e79545-7416-4daa-ac13-d9a0226503e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0aa2c695c43b41ebb8b05ffa370e0a8b', 'request_id': 'req-b8e79545-7416-4daa-ac13-d9a0226503e9'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:05.285900 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b8e79545-7416-4daa-ac13-d9a0226503e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0aa2c695c43b41ebb8b05ffa370e0a8b', 'request_id': 'req-b8e79545-7416-4daa-ac13-d9a0226503e9'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:05.296324 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b8e79545-7416-4daa-ac13-d9a0226503e9 admin admin] The token cache is being invalidate because project 0aa2c695c43b41ebb8b05ffa370e0a8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 09:25:05.296624 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b8e79545-7416-4daa-ac13-d9a0226503e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:05.297244 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b8e79545-7416-4daa-ac13-d9a0226503e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:05.298728 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1307/2599] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 09:25:05 2022] DELETE /identity/v3/projects/0aa2c695c43b41ebb8b05ffa370e0a8b => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:25:05.303801 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9771bb7-8a8c-4051-8f98-641aae21bfc4 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:05.304263 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9771bb7-8a8c-4051-8f98-641aae21bfc4 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:05.304412 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9771bb7-8a8c-4051-8f98-641aae21bfc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:05.324253 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-d9771bb7-8a8c-4051-8f98-641aae21bfc4 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:25:05.325459 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-d9771bb7-8a8c-4051-8f98-641aae21bfc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:25:05.384262 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1293/2600] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:25:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:25:05.392894 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00604257-3a12-4651-85f9-484223f8f005 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:05.393179 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00604257-3a12-4651-85f9-484223f8f005 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:05.393457 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00604257-3a12-4651-85f9-484223f8f005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:05.413133 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-00604257-3a12-4651-85f9-484223f8f005 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:25:05.414363 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-00604257-3a12-4651-85f9-484223f8f005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:25:05.422730 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1308/2601] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:25:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:25:05.426838 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9771bb7-8a8c-4051-8f98-641aae21bfc4 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:05.437155 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9771bb7-8a8c-4051-8f98-641aae21bfc4 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:05.441584 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a08d27d-bbb6-4e56-ae6e-4067676d1d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:05.442212 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08d27d-bbb6-4e56-ae6e-4067676d1d8c admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:05.442491 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08d27d-bbb6-4e56-ae6e-4067676d1d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:05.442764 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08d27d-bbb6-4e56-ae6e-4067676d1d8c admin admin] PATH_INFO: `/v3/domains` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:05.443405 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a08d27d-bbb6-4e56-ae6e-4067676d1d8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 09:25:05.443708 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a08d27d-bbb6-4e56-ae6e-4067676d1d8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:05.445304 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a08d27d-bbb6-4e56-ae6e-4067676d1d8c admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:05.456539 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1294/2602] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 09:25:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 09:25:05.460570 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00604257-3a12-4651-85f9-484223f8f005 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:05.469593 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00604257-3a12-4651-85f9-484223f8f005 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:05.474192 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22da6a17-9172-43ff-a6f3-25f8ff25dcee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:05.474779 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da6a17-9172-43ff-a6f3-25f8ff25dcee admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:05.475062 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da6a17-9172-43ff-a6f3-25f8ff25dcee admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:05.475337 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da6a17-9172-43ff-a6f3-25f8ff25dcee admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:05.475961 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22da6a17-9172-43ff-a6f3-25f8ff25dcee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:05.477865 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22da6a17-9172-43ff-a6f3-25f8ff25dcee admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:05.506067 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1309/2603] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 09:25:05 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:25:05.510421 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a08d27d-bbb6-4e56-ae6e-4067676d1d8c admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:05.520313 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a08d27d-bbb6-4e56-ae6e-4067676d1d8c admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:05.566320 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:25:05.578376 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7206a6a-ecef-4294-be27-dcf07c2cdec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:05.578975 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7206a6a-ecef-4294-be27-dcf07c2cdec9 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:05.579254 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7206a6a-ecef-4294-be27-dcf07c2cdec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:05.579524 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7206a6a-ecef-4294-be27-dcf07c2cdec9 admin admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:05.580152 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7206a6a-ecef-4294-be27-dcf07c2cdec9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:05.582085 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7206a6a-ecef-4294-be27-dcf07c2cdec9 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:05.603026 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1295/2604] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 09:25:05 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 09:25:05.607084 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22da6a17-9172-43ff-a6f3-25f8ff25dcee admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:05.617371 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22da6a17-9172-43ff-a6f3-25f8ff25dcee admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:05.622214 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebca055a-2012-42f9-9e2a-e89aa193d4f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:05.622860 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebca055a-2012-42f9-9e2a-e89aa193d4f5 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:05.623147 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebca055a-2012-42f9-9e2a-e89aa193d4f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:05.623428 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebca055a-2012-42f9-9e2a-e89aa193d4f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:05.624071 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebca055a-2012-42f9-9e2a-e89aa193d4f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:05.625658 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebca055a-2012-42f9-9e2a-e89aa193d4f5 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:05.635801 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1310/2605] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:25:05.640069 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7206a6a-ecef-4294-be27-dcf07c2cdec9 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:05.650598 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7206a6a-ecef-4294-be27-dcf07c2cdec9 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:05.655774 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fe0153-3008-48d1-a60f-3a44a8c18857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:05.656371 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fe0153-3008-48d1-a60f-3a44a8c18857 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:05.656656 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fe0153-3008-48d1-a60f-3a44a8c18857 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:05.656932 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fe0153-3008-48d1-a60f-3a44a8c18857 admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:05.657568 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fe0153-3008-48d1-a60f-3a44a8c18857 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:05.659206 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fe0153-3008-48d1-a60f-3a44a8c18857 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:05.669979 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1296/2606] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:25:05.673873 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebca055a-2012-42f9-9e2a-e89aa193d4f5 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:05.684133 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebca055a-2012-42f9-9e2a-e89aa193d4f5 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:05.689247 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a66b8b0-a8c3-41dd-9e09-38e11068130c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:05.689953 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a66b8b0-a8c3-41dd-9e09-38e11068130c admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:05.690243 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a66b8b0-a8c3-41dd-9e09-38e11068130c admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:05.690526 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a66b8b0-a8c3-41dd-9e09-38e11068130c admin admin] PATH_INFO: `/v3/projects/559ccc6a98b04f338e8e564ded2e93cc/users/56b8becb75524264a90a7bd92690b557/roles/3248f0268738489b883041851d681bb7` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:05.706441 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a66b8b0-a8c3-41dd-9e09-38e11068130c admin admin] RBAC: Authorizing `identity:create_grant(project_id=559ccc6a98b04f338e8e564ded2e93cc, user_id=56b8becb75524264a90a7bd92690b557, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:05.709055 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a66b8b0-a8c3-41dd-9e09-38e11068130c admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:05.721784 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1311/2607] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:25:05 2022] PUT /identity/v3/projects/559ccc6a98b04f338e8e564ded2e93cc/users/56b8becb75524264a90a7bd92690b557/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 09:25:05.731622 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f021674-f325-480b-b6c3-031e09e4ed45 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:05.731905 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f021674-f325-480b-b6c3-031e09e4ed45 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:05.732212 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f021674-f325-480b-b6c3-031e09e4ed45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:05.759538 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-2f021674-f325-480b-b6c3-031e09e4ed45 None None] MFA Rules not processed for user `56b8becb75524264a90a7bd92690b557`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:25:05.760739 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-2f021674-f325-480b-b6c3-031e09e4ed45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:25:05.823162 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1297/2608] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:25:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:25:05.828179 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a66b8b0-a8c3-41dd-9e09-38e11068130c admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:05.858430 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:25:05.890814 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a66b8b0-a8c3-41dd-9e09-38e11068130c admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:05.895267 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d99d430-3286-4633-a224-014a54e627e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:05.895863 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d99d430-3286-4633-a224-014a54e627e9 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:05.896141 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d99d430-3286-4633-a224-014a54e627e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:05.896412 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d99d430-3286-4633-a224-014a54e627e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:05.897031 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d99d430-3286-4633-a224-014a54e627e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:05.898924 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d99d430-3286-4633-a224-014a54e627e9 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:05.930852 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1312/2609] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 09:25:05 2022] POST /identity/v3/projects => generated 373 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:25:05.931953 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f021674-f325-480b-b6c3-031e09e4ed45 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:05.943032 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f021674-f325-480b-b6c3-031e09e4ed45 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:06.000734 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cf99d0-c626-4405-a3ac-b936aca1fc2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:06.001345 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf99d0-c626-4405-a3ac-b936aca1fc2b admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:06.001624 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf99d0-c626-4405-a3ac-b936aca1fc2b admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:06.001928 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf99d0-c626-4405-a3ac-b936aca1fc2b admin admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:06.002576 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cf99d0-c626-4405-a3ac-b936aca1fc2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:06.004608 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cf99d0-c626-4405-a3ac-b936aca1fc2b admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:06.026952 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1298/2610] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 09:25:05 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:25:06.031062 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d99d430-3286-4633-a224-014a54e627e9 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:06.041587 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d99d430-3286-4633-a224-014a54e627e9 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:06.046820 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df476b9-eb37-414f-89bd-129d59b08d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:06.047418 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df476b9-eb37-414f-89bd-129d59b08d92 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:06.047700 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df476b9-eb37-414f-89bd-129d59b08d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:06.047982 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df476b9-eb37-414f-89bd-129d59b08d92 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:06.048621 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df476b9-eb37-414f-89bd-129d59b08d92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:06.050228 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df476b9-eb37-414f-89bd-129d59b08d92 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:06.061785 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1313/2611] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:25:06.065643 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cf99d0-c626-4405-a3ac-b936aca1fc2b admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:06.075464 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cf99d0-c626-4405-a3ac-b936aca1fc2b admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:06.080806 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2ce60c-1669-4a4e-ab9d-9fc48c97e024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:06.081405 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ce60c-1669-4a4e-ab9d-9fc48c97e024 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:06.081708 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ce60c-1669-4a4e-ab9d-9fc48c97e024 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:06.081997 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ce60c-1669-4a4e-ab9d-9fc48c97e024 admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:06.082635 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2ce60c-1669-4a4e-ab9d-9fc48c97e024 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:06.084264 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2ce60c-1669-4a4e-ab9d-9fc48c97e024 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:06.094856 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1299/2612] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:25:06.098578 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df476b9-eb37-414f-89bd-129d59b08d92 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:06.108369 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df476b9-eb37-414f-89bd-129d59b08d92 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:06.112904 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86d5421-da62-4689-894b-8a4ce10582a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:06.113540 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86d5421-da62-4689-894b-8a4ce10582a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:06.113875 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86d5421-da62-4689-894b-8a4ce10582a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:06.114149 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86d5421-da62-4689-894b-8a4ce10582a0 admin admin] PATH_INFO: `/v3/projects/135b0c0977a54e63b6ea34489cb1f3e6/users/efe8fa289f1248be814dc95d06f7f885/roles/3248f0268738489b883041851d681bb7` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:06.128262 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86d5421-da62-4689-894b-8a4ce10582a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=135b0c0977a54e63b6ea34489cb1f3e6, user_id=efe8fa289f1248be814dc95d06f7f885, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:06.130909 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86d5421-da62-4689-894b-8a4ce10582a0 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:06.143796 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1314/2613] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:25:06 2022] PUT /identity/v3/projects/135b0c0977a54e63b6ea34489cb1f3e6/users/efe8fa289f1248be814dc95d06f7f885/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:25:06.154473 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fbe77c-9660-48de-9d2b-42444a3e5b08 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:06.154773 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fbe77c-9660-48de-9d2b-42444a3e5b08 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:06.155047 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fbe77c-9660-48de-9d2b-42444a3e5b08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:06.182431 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-f2fbe77c-9660-48de-9d2b-42444a3e5b08 None None] MFA Rules not processed for user `efe8fa289f1248be814dc95d06f7f885`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:25:06.183627 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-f2fbe77c-9660-48de-9d2b-42444a3e5b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:25:06.210318 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:25:06.250693 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1300/2614] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:25:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:25:06.255266 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86d5421-da62-4689-894b-8a4ce10582a0 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:06.315986 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86d5421-da62-4689-894b-8a4ce10582a0 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:06.320770 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f1a3c-f6fe-4414-bd1e-378642b324fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:06.321370 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f1a3c-f6fe-4414-bd1e-378642b324fa admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:06.321672 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f1a3c-f6fe-4414-bd1e-378642b324fa admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:06.321972 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f1a3c-f6fe-4414-bd1e-378642b324fa admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:06.322638 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464f1a3c-f6fe-4414-bd1e-378642b324fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:06.324516 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464f1a3c-f6fe-4414-bd1e-378642b324fa admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:06.352757 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1315/2615] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 09:25:06 2022] POST /identity/v3/projects => generated 375 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 09:25:06.356808 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fbe77c-9660-48de-9d2b-42444a3e5b08 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:06.367905 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fbe77c-9660-48de-9d2b-42444a3e5b08 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:06.423528 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d326172-6e1b-4682-8a94-b179a4c1ccf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:06.424181 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d326172-6e1b-4682-8a94-b179a4c1ccf8 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:06.425495 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d326172-6e1b-4682-8a94-b179a4c1ccf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:06.425815 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d326172-6e1b-4682-8a94-b179a4c1ccf8 admin admin] PATH_INFO: `/v3/users` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:06.426455 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d326172-6e1b-4682-8a94-b179a4c1ccf8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:06.428644 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d326172-6e1b-4682-8a94-b179a4c1ccf8 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:06.449182 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1301/2616] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 09:25:06 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:25:06.452867 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f1a3c-f6fe-4414-bd1e-378642b324fa admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:06.462887 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f1a3c-f6fe-4414-bd1e-378642b324fa admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:06.467938 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce4dc0a-dd3e-4af5-afb7-9d7b943e7909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:06.468525 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce4dc0a-dd3e-4af5-afb7-9d7b943e7909 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:06.468806 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce4dc0a-dd3e-4af5-afb7-9d7b943e7909 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:06.469080 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce4dc0a-dd3e-4af5-afb7-9d7b943e7909 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:06.469748 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce4dc0a-dd3e-4af5-afb7-9d7b943e7909 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:06.471337 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce4dc0a-dd3e-4af5-afb7-9d7b943e7909 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:06.481530 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1316/2617] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:25:06.486023 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d326172-6e1b-4682-8a94-b179a4c1ccf8 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:06.495992 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d326172-6e1b-4682-8a94-b179a4c1ccf8 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:06.501344 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2556b80-fbd0-4fd6-89ce-37b93c113e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:06.502022 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2556b80-fbd0-4fd6-89ce-37b93c113e16 admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:06.502307 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2556b80-fbd0-4fd6-89ce-37b93c113e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:06.502582 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2556b80-fbd0-4fd6-89ce-37b93c113e16 admin admin] PATH_INFO: `/v3/projects/fd0e362054be486ea7646280bfdc9850/users/e1d9bbf40bbe496bab550762a21de890/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:06.517943 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2556b80-fbd0-4fd6-89ce-37b93c113e16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd0e362054be486ea7646280bfdc9850, user_id=e1d9bbf40bbe496bab550762a21de890, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:06.522217 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2556b80-fbd0-4fd6-89ce-37b93c113e16 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:06.536846 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1302/2618] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:25:06 2022] PUT /identity/v3/projects/fd0e362054be486ea7646280bfdc9850/users/e1d9bbf40bbe496bab550762a21de890/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:25:06.540777 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce4dc0a-dd3e-4af5-afb7-9d7b943e7909 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:06.550727 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce4dc0a-dd3e-4af5-afb7-9d7b943e7909 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:06.598022 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:25:06.607659 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5dece2-bdde-4137-9747-d9a844889902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:06.608263 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5dece2-bdde-4137-9747-d9a844889902 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:06.608549 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5dece2-bdde-4137-9747-d9a844889902 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:06.608825 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5dece2-bdde-4137-9747-d9a844889902 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:06.609457 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5dece2-bdde-4137-9747-d9a844889902 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:06.611091 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5dece2-bdde-4137-9747-d9a844889902 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:06.620764 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1317/2619] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:25:06.625652 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2556b80-fbd0-4fd6-89ce-37b93c113e16 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:06.635855 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2556b80-fbd0-4fd6-89ce-37b93c113e16 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:06.641008 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef75e0f-df92-463d-bc66-bb9acd9514ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:06.641653 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef75e0f-df92-463d-bc66-bb9acd9514ce admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:06.641978 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef75e0f-df92-463d-bc66-bb9acd9514ce admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:06.642243 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef75e0f-df92-463d-bc66-bb9acd9514ce admin admin] PATH_INFO: `/v3/projects/fd0e362054be486ea7646280bfdc9850/users/e1d9bbf40bbe496bab550762a21de890/roles/14615e30ed3f4d8cb78ee5e0bf9b5761` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:06.644181 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef75e0f-df92-463d-bc66-bb9acd9514ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd0e362054be486ea7646280bfdc9850, user_id=e1d9bbf40bbe496bab550762a21de890, role_id=14615e30ed3f4d8cb78ee5e0bf9b5761)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:06.646854 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef75e0f-df92-463d-bc66-bb9acd9514ce admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:06.657079 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1303/2620] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:25:06 2022] PUT /identity/v3/projects/fd0e362054be486ea7646280bfdc9850/users/e1d9bbf40bbe496bab550762a21de890/roles/14615e30ed3f4d8cb78ee5e0bf9b5761 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:25:06.667790 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e1387-a279-46ec-95db-06915572071b None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:06.668092 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e1387-a279-46ec-95db-06915572071b None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:06.668374 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e1387-a279-46ec-95db-06915572071b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:06.695982 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-bc2e1387-a279-46ec-95db-06915572071b None None] MFA Rules not processed for user `e1d9bbf40bbe496bab550762a21de890`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:25:06.697178 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-bc2e1387-a279-46ec-95db-06915572071b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:25:06.765935 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1318/2621] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:25:06 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:25:06.777385 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef75e0f-df92-463d-bc66-bb9acd9514ce admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:06.779641 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-6ef75e0f-df92-463d-bc66-bb9acd9514ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:25:06.839149 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef75e0f-df92-463d-bc66-bb9acd9514ce admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:06.843050 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020d7f6b-fcbd-492e-8f60-c36686cd0188 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:06.843655 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d7f6b-fcbd-492e-8f60-c36686cd0188 service cinder] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:06.843941 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d7f6b-fcbd-492e-8f60-c36686cd0188 service cinder] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:06.844217 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d7f6b-fcbd-492e-8f60-c36686cd0188 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:06.877040 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:25:06.901850 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020d7f6b-fcbd-492e-8f60-c36686cd0188 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:06.903292 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020d7f6b-fcbd-492e-8f60-c36686cd0188 service cinder] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:06.912661 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1304/2622] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:25:06 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:25:08.408583 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2e1387-a279-46ec-95db-06915572071b None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:08.410078 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-bc2e1387-a279-46ec-95db-06915572071b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:25:08.423053 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2e1387-a279-46ec-95db-06915572071b None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:08.426645 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ada79e-2b2e-4eb5-b53c-a65a0145af11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:08.428144 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ada79e-2b2e-4eb5-b53c-a65a0145af11 service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:08.428428 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ada79e-2b2e-4eb5-b53c-a65a0145af11 service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:08.428699 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ada79e-2b2e-4eb5-b53c-a65a0145af11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:08.485457 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ada79e-2b2e-4eb5-b53c-a65a0145af11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:08.486926 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ada79e-2b2e-4eb5-b53c-a65a0145af11 service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:08.495638 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1319/2623] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:25:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:25:08.997717 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020d7f6b-fcbd-492e-8f60-c36686cd0188 service cinder] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:09.006214 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020d7f6b-fcbd-492e-8f60-c36686cd0188 service cinder] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:09.009767 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc23bd77-0f26-4498-80b2-2a9a1f0d45ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:09.010364 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc23bd77-0f26-4498-80b2-2a9a1f0d45ce service cinder] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:09.010644 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc23bd77-0f26-4498-80b2-2a9a1f0d45ce service cinder] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:09.010918 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc23bd77-0f26-4498-80b2-2a9a1f0d45ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:09.019895 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc23bd77-0f26-4498-80b2-2a9a1f0d45ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:09.021346 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc23bd77-0f26-4498-80b2-2a9a1f0d45ce service cinder] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:09.031328 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1305/2624] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:25:08 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:25:12.873331 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ada79e-2b2e-4eb5-b53c-a65a0145af11 service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:12.934151 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ada79e-2b2e-4eb5-b53c-a65a0145af11 service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:12.939152 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e9260e-4b0e-40f5-bc87-63ba6ee0f419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:12.939775 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e9260e-4b0e-40f5-bc87-63ba6ee0f419 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:12.940056 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e9260e-4b0e-40f5-bc87-63ba6ee0f419 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:12.940331 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e9260e-4b0e-40f5-bc87-63ba6ee0f419 admin admin] PATH_INFO: `/v3/users/56b8becb75524264a90a7bd92690b557` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:12.941436 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e9260e-4b0e-40f5-bc87-63ba6ee0f419 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56b8becb75524264a90a7bd92690b557)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:12.943374 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e9260e-4b0e-40f5-bc87-63ba6ee0f419 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:12.957992 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:25:12.991301 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-a7e9260e-4b0e-40f5-bc87-63ba6ee0f419 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56b8becb75524264a90a7bd92690b557'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:12.991945 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-a7e9260e-4b0e-40f5-bc87-63ba6ee0f419 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56b8becb75524264a90a7bd92690b557'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:12.998787 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-a7e9260e-4b0e-40f5-bc87-63ba6ee0f419 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56b8becb75524264a90a7bd92690b557'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:13.010336 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-a7e9260e-4b0e-40f5-bc87-63ba6ee0f419 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56b8becb75524264a90a7bd92690b557'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:13.017957 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-a7e9260e-4b0e-40f5-bc87-63ba6ee0f419 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56b8becb75524264a90a7bd92690b557'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:13.024103 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1320/2625] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 09:25:12 2022] DELETE /identity/v3/users/56b8becb75524264a90a7bd92690b557 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:25:13.028325 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b03bf29-7430-4189-89fa-49d2d50f94eb None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:13.029503 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b03bf29-7430-4189-89fa-49d2d50f94eb None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:13.029618 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b03bf29-7430-4189-89fa-49d2d50f94eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:13.047927 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-6b03bf29-7430-4189-89fa-49d2d50f94eb None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:25:13.049121 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-6b03bf29-7430-4189-89fa-49d2d50f94eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:25:13.107674 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1306/2626] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:25:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:25:13.117192 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e9260e-4b0e-40f5-bc87-63ba6ee0f419 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:13.118595 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-a7e9260e-4b0e-40f5-bc87-63ba6ee0f419 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:25:13.174569 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e9260e-4b0e-40f5-bc87-63ba6ee0f419 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:13.178408 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400c8cb5-82f6-48ea-8d42-1114feed7228 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:13.178999 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400c8cb5-82f6-48ea-8d42-1114feed7228 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:13.179275 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400c8cb5-82f6-48ea-8d42-1114feed7228 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:13.179544 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400c8cb5-82f6-48ea-8d42-1114feed7228 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:13.188713 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400c8cb5-82f6-48ea-8d42-1114feed7228 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:13.190151 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400c8cb5-82f6-48ea-8d42-1114feed7228 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:13.199452 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1321/2627] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:25:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:25:13.461596 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b03bf29-7430-4189-89fa-49d2d50f94eb None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:13.463061 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-6b03bf29-7430-4189-89fa-49d2d50f94eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:25:13.476380 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b03bf29-7430-4189-89fa-49d2d50f94eb None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:13.480986 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d08040-5be1-485c-8b35-dc9200ee8553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:13.482232 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d08040-5be1-485c-8b35-dc9200ee8553 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:13.482528 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d08040-5be1-485c-8b35-dc9200ee8553 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:13.482972 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d08040-5be1-485c-8b35-dc9200ee8553 admin admin] PATH_INFO: `/v3/projects/559ccc6a98b04f338e8e564ded2e93cc` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:13.484090 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d08040-5be1-485c-8b35-dc9200ee8553 admin admin] RBAC: Authorizing `identity:delete_project(project_id=559ccc6a98b04f338e8e564ded2e93cc)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:13.486046 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d08040-5be1-485c-8b35-dc9200ee8553 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:13.532885 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-30d08040-5be1-485c-8b35-dc9200ee8553 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '559ccc6a98b04f338e8e564ded2e93cc', 'request_id': 'req-30d08040-5be1-485c-8b35-dc9200ee8553'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:13.539088 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-30d08040-5be1-485c-8b35-dc9200ee8553 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '559ccc6a98b04f338e8e564ded2e93cc', 'request_id': 'req-30d08040-5be1-485c-8b35-dc9200ee8553'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:13.546257 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-30d08040-5be1-485c-8b35-dc9200ee8553 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '559ccc6a98b04f338e8e564ded2e93cc', 'request_id': 'req-30d08040-5be1-485c-8b35-dc9200ee8553'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:13.556882 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-30d08040-5be1-485c-8b35-dc9200ee8553 admin admin] The token cache is being invalidate because project 559ccc6a98b04f338e8e564ded2e93cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 09:25:13.557171 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-30d08040-5be1-485c-8b35-dc9200ee8553 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:13.557812 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-30d08040-5be1-485c-8b35-dc9200ee8553 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:13.559281 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1307/2628] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 09:25:13 2022] DELETE /identity/v3/projects/559ccc6a98b04f338e8e564ded2e93cc => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:25:13.563881 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400c8cb5-82f6-48ea-8d42-1114feed7228 service neutron] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:13.573360 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400c8cb5-82f6-48ea-8d42-1114feed7228 service neutron] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:13.628484 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faa370a-21c1-46da-b62f-8111bcf64ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:13.629102 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa370a-21c1-46da-b62f-8111bcf64ef3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:13.629385 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa370a-21c1-46da-b62f-8111bcf64ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:13.629661 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa370a-21c1-46da-b62f-8111bcf64ef3 admin admin] PATH_INFO: `/v3/users/efe8fa289f1248be814dc95d06f7f885` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:13.630801 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faa370a-21c1-46da-b62f-8111bcf64ef3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=efe8fa289f1248be814dc95d06f7f885)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:13.632760 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faa370a-21c1-46da-b62f-8111bcf64ef3 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:13.668845 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:25:13.679082 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-2faa370a-21c1-46da-b62f-8111bcf64ef3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efe8fa289f1248be814dc95d06f7f885'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:13.679860 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-2faa370a-21c1-46da-b62f-8111bcf64ef3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efe8fa289f1248be814dc95d06f7f885'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:13.687180 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-2faa370a-21c1-46da-b62f-8111bcf64ef3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efe8fa289f1248be814dc95d06f7f885'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:13.698749 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-2faa370a-21c1-46da-b62f-8111bcf64ef3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efe8fa289f1248be814dc95d06f7f885'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:13.705914 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-2faa370a-21c1-46da-b62f-8111bcf64ef3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efe8fa289f1248be814dc95d06f7f885'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:13.711810 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1322/2629] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 09:25:13 2022] DELETE /identity/v3/users/efe8fa289f1248be814dc95d06f7f885 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:25:13.973426 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d08040-5be1-485c-8b35-dc9200ee8553 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:13.974961 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-30d08040-5be1-485c-8b35-dc9200ee8553 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:25:13.988229 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d08040-5be1-485c-8b35-dc9200ee8553 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:14.029949 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:25:14.046642 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bf5259-9805-4860-a0d0-7dd61c224eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:14.047281 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf5259-9805-4860-a0d0-7dd61c224eb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:14.047566 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf5259-9805-4860-a0d0-7dd61c224eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:14.047874 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf5259-9805-4860-a0d0-7dd61c224eb7 admin admin] PATH_INFO: `/v3/projects/135b0c0977a54e63b6ea34489cb1f3e6` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:14.049001 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bf5259-9805-4860-a0d0-7dd61c224eb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=135b0c0977a54e63b6ea34489cb1f3e6)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:14.050968 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bf5259-9805-4860-a0d0-7dd61c224eb7 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:14.097323 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-35bf5259-9805-4860-a0d0-7dd61c224eb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '135b0c0977a54e63b6ea34489cb1f3e6', 'request_id': 'req-35bf5259-9805-4860-a0d0-7dd61c224eb7'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:14.103504 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-35bf5259-9805-4860-a0d0-7dd61c224eb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '135b0c0977a54e63b6ea34489cb1f3e6', 'request_id': 'req-35bf5259-9805-4860-a0d0-7dd61c224eb7'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:14.111664 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-35bf5259-9805-4860-a0d0-7dd61c224eb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '135b0c0977a54e63b6ea34489cb1f3e6', 'request_id': 'req-35bf5259-9805-4860-a0d0-7dd61c224eb7'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:14.122095 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-35bf5259-9805-4860-a0d0-7dd61c224eb7 admin admin] The token cache is being invalidate because project 135b0c0977a54e63b6ea34489cb1f3e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 09:25:14.122384 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-35bf5259-9805-4860-a0d0-7dd61c224eb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:14.123060 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-35bf5259-9805-4860-a0d0-7dd61c224eb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:14.124545 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1308/2630] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 09:25:13 2022] DELETE /identity/v3/projects/135b0c0977a54e63b6ea34489cb1f3e6 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:25:14.128753 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faa370a-21c1-46da-b62f-8111bcf64ef3 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:14.130168 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-2faa370a-21c1-46da-b62f-8111bcf64ef3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:25:14.142550 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faa370a-21c1-46da-b62f-8111bcf64ef3 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:14.197136 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213f8e51-6479-4e17-be96-b925c2eef828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:14.197776 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f8e51-6479-4e17-be96-b925c2eef828 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:14.198056 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f8e51-6479-4e17-be96-b925c2eef828 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:14.198329 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f8e51-6479-4e17-be96-b925c2eef828 admin admin] PATH_INFO: `/v3/users/e1d9bbf40bbe496bab550762a21de890` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:14.199423 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213f8e51-6479-4e17-be96-b925c2eef828 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1d9bbf40bbe496bab550762a21de890)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:14.201347 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213f8e51-6479-4e17-be96-b925c2eef828 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:14.246305 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-213f8e51-6479-4e17-be96-b925c2eef828 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1d9bbf40bbe496bab550762a21de890'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:14.246946 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-213f8e51-6479-4e17-be96-b925c2eef828 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1d9bbf40bbe496bab550762a21de890'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:14.253852 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-213f8e51-6479-4e17-be96-b925c2eef828 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1d9bbf40bbe496bab550762a21de890'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:14.265337 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-213f8e51-6479-4e17-be96-b925c2eef828 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1d9bbf40bbe496bab550762a21de890'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:14.272246 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-213f8e51-6479-4e17-be96-b925c2eef828 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1d9bbf40bbe496bab550762a21de890'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:14.278039 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1323/2631] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 09:25:14 2022] DELETE /identity/v3/users/e1d9bbf40bbe496bab550762a21de890 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:25:14.542171 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bf5259-9805-4860-a0d0-7dd61c224eb7 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:14.543608 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-35bf5259-9805-4860-a0d0-7dd61c224eb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:25:14.556829 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bf5259-9805-4860-a0d0-7dd61c224eb7 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:14.613357 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2568dbe-89e0-4205-876a-549d2324216b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:14.614018 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2568dbe-89e0-4205-876a-549d2324216b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:14.614303 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2568dbe-89e0-4205-876a-549d2324216b admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:14.614580 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2568dbe-89e0-4205-876a-549d2324216b admin admin] PATH_INFO: `/v3/projects/fd0e362054be486ea7646280bfdc9850` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:14.615681 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2568dbe-89e0-4205-876a-549d2324216b admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd0e362054be486ea7646280bfdc9850)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:14.617582 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2568dbe-89e0-4205-876a-549d2324216b admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:14.663188 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-e2568dbe-89e0-4205-876a-549d2324216b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd0e362054be486ea7646280bfdc9850', 'request_id': 'req-e2568dbe-89e0-4205-876a-549d2324216b'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:14.669219 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-e2568dbe-89e0-4205-876a-549d2324216b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd0e362054be486ea7646280bfdc9850', 'request_id': 'req-e2568dbe-89e0-4205-876a-549d2324216b'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:14.676387 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-e2568dbe-89e0-4205-876a-549d2324216b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd0e362054be486ea7646280bfdc9850', 'request_id': 'req-e2568dbe-89e0-4205-876a-549d2324216b'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:14.687695 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-e2568dbe-89e0-4205-876a-549d2324216b admin admin] The token cache is being invalidate because project fd0e362054be486ea7646280bfdc9850 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 09:25:14.687968 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-e2568dbe-89e0-4205-876a-549d2324216b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:14.688592 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-e2568dbe-89e0-4205-876a-549d2324216b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:14.690113 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1309/2632] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 09:25:14 2022] DELETE /identity/v3/projects/fd0e362054be486ea7646280bfdc9850 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:25:14.695061 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eded62e8-4615-461f-8ee3-244fb3bf789c None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:14.695349 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eded62e8-4615-461f-8ee3-244fb3bf789c None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:14.695655 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eded62e8-4615-461f-8ee3-244fb3bf789c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:14.715273 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-eded62e8-4615-461f-8ee3-244fb3bf789c None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:25:14.716496 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-eded62e8-4615-461f-8ee3-244fb3bf789c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:25:14.746714 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:25:14.778571 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1324/2633] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:25:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:25:14.786490 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c84356-880d-42a2-9aed-0ff393d22443 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:14.786722 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c84356-880d-42a2-9aed-0ff393d22443 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:14.786998 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c84356-880d-42a2-9aed-0ff393d22443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:14.795349 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:25:14.807429 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-73c84356-880d-42a2-9aed-0ff393d22443 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:25:14.808629 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-73c84356-880d-42a2-9aed-0ff393d22443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:25:14.816967 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1310/2634] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:25:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:25:14.821159 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eded62e8-4615-461f-8ee3-244fb3bf789c None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:14.831464 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eded62e8-4615-461f-8ee3-244fb3bf789c None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:14.836026 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b728aa-c92d-4c2c-a052-f7876f38cc8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:14.836622 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b728aa-c92d-4c2c-a052-f7876f38cc8c admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:14.836908 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b728aa-c92d-4c2c-a052-f7876f38cc8c admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:14.837186 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b728aa-c92d-4c2c-a052-f7876f38cc8c admin admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:14.837864 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b728aa-c92d-4c2c-a052-f7876f38cc8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 09:25:14.838173 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b728aa-c92d-4c2c-a052-f7876f38cc8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:14.839771 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b728aa-c92d-4c2c-a052-f7876f38cc8c admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:14.851489 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1325/2635] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 09:25:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 09:25:14.855674 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c84356-880d-42a2-9aed-0ff393d22443 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:14.864866 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c84356-880d-42a2-9aed-0ff393d22443 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:14.869417 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48813fbc-5739-4ac5-b44f-cd8687fd5e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:14.870038 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48813fbc-5739-4ac5-b44f-cd8687fd5e8c admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:14.870318 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48813fbc-5739-4ac5-b44f-cd8687fd5e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:14.870589 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48813fbc-5739-4ac5-b44f-cd8687fd5e8c admin admin] PATH_INFO: `/v3/projects` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:14.871218 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48813fbc-5739-4ac5-b44f-cd8687fd5e8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:14.873644 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48813fbc-5739-4ac5-b44f-cd8687fd5e8c admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:14.903123 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1311/2636] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 09:25:14 2022] POST /identity/v3/projects => generated 369 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:25:14.904239 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b728aa-c92d-4c2c-a052-f7876f38cc8c admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:14.914420 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b728aa-c92d-4c2c-a052-f7876f38cc8c admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:14.970800 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d05c53e-d1a9-4654-ace9-3e884d9fef73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:14.971397 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d05c53e-d1a9-4654-ace9-3e884d9fef73 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:14.971683 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d05c53e-d1a9-4654-ace9-3e884d9fef73 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:14.971962 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d05c53e-d1a9-4654-ace9-3e884d9fef73 admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:14.972587 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d05c53e-d1a9-4654-ace9-3e884d9fef73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:14.974525 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d05c53e-d1a9-4654-ace9-3e884d9fef73 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:14.995280 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1326/2637] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 09:25:14 2022] POST /identity/v3/users => generated 285 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 09:25:15.000080 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48813fbc-5739-4ac5-b44f-cd8687fd5e8c admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:15.010360 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48813fbc-5739-4ac5-b44f-cd8687fd5e8c admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:15.015539 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a532895-7e5d-4496-bf4d-7e7b0574ff8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:15.016140 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a532895-7e5d-4496-bf4d-7e7b0574ff8e admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:15.016421 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a532895-7e5d-4496-bf4d-7e7b0574ff8e admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:15.016696 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a532895-7e5d-4496-bf4d-7e7b0574ff8e admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:15.017334 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a532895-7e5d-4496-bf4d-7e7b0574ff8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:15.018971 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a532895-7e5d-4496-bf4d-7e7b0574ff8e admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:15.028847 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1312/2638] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:25:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:25:15.032714 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d05c53e-d1a9-4654-ace9-3e884d9fef73 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:15.042740 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d05c53e-d1a9-4654-ace9-3e884d9fef73 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:15.047191 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd42f147-c02e-4b7f-a58e-dd53f17101be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:15.047779 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd42f147-c02e-4b7f-a58e-dd53f17101be admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:15.048063 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd42f147-c02e-4b7f-a58e-dd53f17101be admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:15.048337 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd42f147-c02e-4b7f-a58e-dd53f17101be admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:15.048965 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd42f147-c02e-4b7f-a58e-dd53f17101be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:15.050557 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd42f147-c02e-4b7f-a58e-dd53f17101be admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:15.061672 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1327/2639] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:25:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:25:15.065813 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a532895-7e5d-4496-bf4d-7e7b0574ff8e admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:15.076296 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a532895-7e5d-4496-bf4d-7e7b0574ff8e admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:15.080910 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570614fb-6dbc-41b8-bdc5-9452666ce11e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:15.081556 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570614fb-6dbc-41b8-bdc5-9452666ce11e admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:15.081871 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570614fb-6dbc-41b8-bdc5-9452666ce11e admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:15.082148 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570614fb-6dbc-41b8-bdc5-9452666ce11e admin admin] PATH_INFO: `/v3/projects/0af054e921664caa8e4b6df9d71d5551/users/e4a086ffca2640db8c3e5566c002ac5f/roles/3248f0268738489b883041851d681bb7` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:15.097483 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570614fb-6dbc-41b8-bdc5-9452666ce11e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0af054e921664caa8e4b6df9d71d5551, user_id=e4a086ffca2640db8c3e5566c002ac5f, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:15.100163 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570614fb-6dbc-41b8-bdc5-9452666ce11e admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:15.113581 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1313/2640] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:25:15 2022] PUT /identity/v3/projects/0af054e921664caa8e4b6df9d71d5551/users/e4a086ffca2640db8c3e5566c002ac5f/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:25:15.188192 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ff33f4-e411-4b15-946d-cf9a77dbbd6d None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:15.188779 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ff33f4-e411-4b15-946d-cf9a77dbbd6d None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:15.188779 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ff33f4-e411-4b15-946d-cf9a77dbbd6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:15.216211 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-80ff33f4-e411-4b15-946d-cf9a77dbbd6d None None] MFA Rules not processed for user `e4a086ffca2640db8c3e5566c002ac5f`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:25:15.217426 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-80ff33f4-e411-4b15-946d-cf9a77dbbd6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:25:15.280118 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1328/2641] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:25:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:25:15.292470 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570614fb-6dbc-41b8-bdc5-9452666ce11e admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:15.294807 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-570614fb-6dbc-41b8-bdc5-9452666ce11e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:25:15.350728 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570614fb-6dbc-41b8-bdc5-9452666ce11e admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:15.354299 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253ee155-9bf1-4042-8a78-f34db5e4d4b1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71692d05fd324679b1598feb52b3fe6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:15.354892 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ee155-9bf1-4042-8a78-f34db5e4d4b1 service glance] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:15.355168 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ee155-9bf1-4042-8a78-f34db5e4d4b1 service glance] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:15.355437 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ee155-9bf1-4042-8a78-f34db5e4d4b1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:15.363993 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253ee155-9bf1-4042-8a78-f34db5e4d4b1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:15.365405 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253ee155-9bf1-4042-8a78-f34db5e4d4b1 service glance] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:15.374576 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1314/2642] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:25:15 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:25:25.277239 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=84,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:25:25.370029 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=51,keystone:INSERT=2 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:25:46.567832 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ff33f4-e411-4b15-946d-cf9a77dbbd6d None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:46.629441 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ff33f4-e411-4b15-946d-cf9a77dbbd6d None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:46.633932 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c347b0-48cf-4ccc-87a4-a19f304176d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:46.634560 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c347b0-48cf-4ccc-87a4-a19f304176d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:46.634841 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c347b0-48cf-4ccc-87a4-a19f304176d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:46.635118 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c347b0-48cf-4ccc-87a4-a19f304176d5 admin admin] PATH_INFO: `/v3/users/e4a086ffca2640db8c3e5566c002ac5f` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:46.636228 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c347b0-48cf-4ccc-87a4-a19f304176d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4a086ffca2640db8c3e5566c002ac5f)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:46.638177 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c347b0-48cf-4ccc-87a4-a19f304176d5 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:46.683242 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-26c347b0-48cf-4ccc-87a4-a19f304176d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4a086ffca2640db8c3e5566c002ac5f'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:46.683814 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-26c347b0-48cf-4ccc-87a4-a19f304176d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4a086ffca2640db8c3e5566c002ac5f'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:46.691418 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-26c347b0-48cf-4ccc-87a4-a19f304176d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4a086ffca2640db8c3e5566c002ac5f'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:46.703052 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-26c347b0-48cf-4ccc-87a4-a19f304176d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4a086ffca2640db8c3e5566c002ac5f'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:46.710030 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-26c347b0-48cf-4ccc-87a4-a19f304176d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4a086ffca2640db8c3e5566c002ac5f'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:46.715894 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1329/2643] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 09:25:46 2022] DELETE /identity/v3/users/e4a086ffca2640db8c3e5566c002ac5f => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:25:46.720504 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bced1d-9f5e-47e0-9e04-b2501b2cace3 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:46.721012 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bced1d-9f5e-47e0-9e04-b2501b2cace3 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:46.721122 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bced1d-9f5e-47e0-9e04-b2501b2cace3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:46.740158 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-c4bced1d-9f5e-47e0-9e04-b2501b2cace3 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:25:46.741360 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-c4bced1d-9f5e-47e0-9e04-b2501b2cace3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:25:46.797635 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1315/2644] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:25:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 09:25:46.808074 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c347b0-48cf-4ccc-87a4-a19f304176d5 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:46.809465 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-26c347b0-48cf-4ccc-87a4-a19f304176d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:25:46.867151 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c347b0-48cf-4ccc-87a4-a19f304176d5 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:46.870871 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1420d0a0-2bd1-428e-8a36-6a44bb012cca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:46.871477 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1420d0a0-2bd1-428e-8a36-6a44bb012cca service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:46.871759 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1420d0a0-2bd1-428e-8a36-6a44bb012cca service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:46.872036 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1420d0a0-2bd1-428e-8a36-6a44bb012cca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:46.880849 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1420d0a0-2bd1-428e-8a36-6a44bb012cca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:46.882300 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1420d0a0-2bd1-428e-8a36-6a44bb012cca service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:46.891511 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1330/2645] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:25:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:25:47.153224 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bced1d-9f5e-47e0-9e04-b2501b2cace3 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:47.154732 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-c4bced1d-9f5e-47e0-9e04-b2501b2cace3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:25:47.167840 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bced1d-9f5e-47e0-9e04-b2501b2cace3 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:47.172468 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3809f8ae-a275-4fa5-826b-1377e5f8351c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:47.173112 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3809f8ae-a275-4fa5-826b-1377e5f8351c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:47.173988 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3809f8ae-a275-4fa5-826b-1377e5f8351c admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:47.174265 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3809f8ae-a275-4fa5-826b-1377e5f8351c admin admin] PATH_INFO: `/v3/projects/0af054e921664caa8e4b6df9d71d5551` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:47.175470 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3809f8ae-a275-4fa5-826b-1377e5f8351c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0af054e921664caa8e4b6df9d71d5551)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:47.177424 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3809f8ae-a275-4fa5-826b-1377e5f8351c admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:47.224619 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-3809f8ae-a275-4fa5-826b-1377e5f8351c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0af054e921664caa8e4b6df9d71d5551', 'request_id': 'req-3809f8ae-a275-4fa5-826b-1377e5f8351c'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:47.231020 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-3809f8ae-a275-4fa5-826b-1377e5f8351c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0af054e921664caa8e4b6df9d71d5551', 'request_id': 'req-3809f8ae-a275-4fa5-826b-1377e5f8351c'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:47.237943 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-3809f8ae-a275-4fa5-826b-1377e5f8351c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0af054e921664caa8e4b6df9d71d5551', 'request_id': 'req-3809f8ae-a275-4fa5-826b-1377e5f8351c'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:47.248035 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-3809f8ae-a275-4fa5-826b-1377e5f8351c admin admin] The token cache is being invalidate because project 0af054e921664caa8e4b6df9d71d5551 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 09:25:47.248319 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-3809f8ae-a275-4fa5-826b-1377e5f8351c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:47.248931 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-3809f8ae-a275-4fa5-826b-1377e5f8351c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:47.250434 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1316/2646] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 09:25:47 2022] DELETE /identity/v3/projects/0af054e921664caa8e4b6df9d71d5551 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:25:47.255275 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d48d17f-05eb-46b3-af27-33cefda59a43 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:47.255561 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d48d17f-05eb-46b3-af27-33cefda59a43 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:47.255835 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d48d17f-05eb-46b3-af27-33cefda59a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:47.275464 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-7d48d17f-05eb-46b3-af27-33cefda59a43 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:25:47.276644 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-7d48d17f-05eb-46b3-af27-33cefda59a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:25:47.293791 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:25:47.336617 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1331/2647] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:25:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:25:47.344376 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511317b0-8438-4522-a9eb-e02fa5bdd6f6 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:47.344658 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511317b0-8438-4522-a9eb-e02fa5bdd6f6 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:47.344934 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511317b0-8438-4522-a9eb-e02fa5bdd6f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:47.364993 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-511317b0-8438-4522-a9eb-e02fa5bdd6f6 None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:25:47.366282 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-511317b0-8438-4522-a9eb-e02fa5bdd6f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:25:47.374759 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1317/2648] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:25:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:25:47.379003 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d48d17f-05eb-46b3-af27-33cefda59a43 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:47.388834 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d48d17f-05eb-46b3-af27-33cefda59a43 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:47.393976 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036a0964-cb9c-4508-b827-3fda67cfb314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:47.394581 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a0964-cb9c-4508-b827-3fda67cfb314 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:47.394860 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a0964-cb9c-4508-b827-3fda67cfb314 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:47.395135 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a0964-cb9c-4508-b827-3fda67cfb314 admin admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:47.395779 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036a0964-cb9c-4508-b827-3fda67cfb314 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 09:25:47.396081 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036a0964-cb9c-4508-b827-3fda67cfb314 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:47.397662 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036a0964-cb9c-4508-b827-3fda67cfb314 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:47.409872 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1332/2649] 10.222.0.30 () {58 vars in 1153 bytes} [Sat Oct 15 09:25:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 09:25:47.414199 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511317b0-8438-4522-a9eb-e02fa5bdd6f6 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:47.423291 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511317b0-8438-4522-a9eb-e02fa5bdd6f6 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:47.427854 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a773efa3-8789-4ff1-bded-e92115005fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:47.428446 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a773efa3-8789-4ff1-bded-e92115005fd7 admin admin] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:47.428980 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a773efa3-8789-4ff1-bded-e92115005fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:47.429252 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a773efa3-8789-4ff1-bded-e92115005fd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:47.429912 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a773efa3-8789-4ff1-bded-e92115005fd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:47.431794 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a773efa3-8789-4ff1-bded-e92115005fd7 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:47.458329 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1318/2650] 10.222.0.30 () {60 vars in 1153 bytes} [Sat Oct 15 09:25:47 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 09:25:47.462462 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036a0964-cb9c-4508-b827-3fda67cfb314 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:47.472976 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036a0964-cb9c-4508-b827-3fda67cfb314 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:47.527759 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c3f66d-a978-458d-b601-91a7e69b24f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:47.528364 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c3f66d-a978-458d-b601-91a7e69b24f9 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:47.528648 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c3f66d-a978-458d-b601-91a7e69b24f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:47.528925 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c3f66d-a978-458d-b601-91a7e69b24f9 admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:47.529560 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c3f66d-a978-458d-b601-91a7e69b24f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:47.531502 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c3f66d-a978-458d-b601-91a7e69b24f9 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:47.556734 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1333/2651] 10.222.0.30 () {60 vars in 1144 bytes} [Sat Oct 15 09:25:47 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 09:25:47.558077 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a773efa3-8789-4ff1-bded-e92115005fd7 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:47.568211 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a773efa3-8789-4ff1-bded-e92115005fd7 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:47.572667 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff05ec2-8bd1-40b7-a3a6-39c1f8e4af05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:47.573246 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff05ec2-8bd1-40b7-a3a6-39c1f8e4af05 admin admin] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:47.573525 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff05ec2-8bd1-40b7-a3a6-39c1f8e4af05 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:47.573831 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff05ec2-8bd1-40b7-a3a6-39c1f8e4af05 admin admin] PATH_INFO: `/v3/roles` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:47.574464 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff05ec2-8bd1-40b7-a3a6-39c1f8e4af05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:47.576083 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff05ec2-8bd1-40b7-a3a6-39c1f8e4af05 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:47.586357 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1319/2652] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:25:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:25:47.590150 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c3f66d-a978-458d-b601-91a7e69b24f9 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:47.600392 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c3f66d-a978-458d-b601-91a7e69b24f9 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:47.605173 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc095f23-6a03-4261-909e-1a48ac703158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:47.605798 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc095f23-6a03-4261-909e-1a48ac703158 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:47.606086 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc095f23-6a03-4261-909e-1a48ac703158 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:47.606365 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc095f23-6a03-4261-909e-1a48ac703158 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:47.607002 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc095f23-6a03-4261-909e-1a48ac703158 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:47.608585 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc095f23-6a03-4261-909e-1a48ac703158 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:47.619170 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1334/2653] 10.222.0.30 () {58 vars in 1122 bytes} [Sat Oct 15 09:25:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 09:25:47.622794 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff05ec2-8bd1-40b7-a3a6-39c1f8e4af05 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:47.632910 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff05ec2-8bd1-40b7-a3a6-39c1f8e4af05 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:47.637436 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddb0be7-edcf-46ad-b9a5-dff9634aa78f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:47.638097 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddb0be7-edcf-46ad-b9a5-dff9634aa78f admin admin] REQUEST_METHOD: `PUT` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:47.638376 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddb0be7-edcf-46ad-b9a5-dff9634aa78f admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:47.638649 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddb0be7-edcf-46ad-b9a5-dff9634aa78f admin admin] PATH_INFO: `/v3/projects/3e406b00fdd045668989289a7bf991d3/users/19d5eb7b7dda46d39b055bc465893ccc/roles/3248f0268738489b883041851d681bb7` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:47.653268 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddb0be7-edcf-46ad-b9a5-dff9634aa78f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e406b00fdd045668989289a7bf991d3, user_id=19d5eb7b7dda46d39b055bc465893ccc, role_id=3248f0268738489b883041851d681bb7)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:47.656011 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddb0be7-edcf-46ad-b9a5-dff9634aa78f admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:47.668470 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1320/2654] 10.222.0.30 () {60 vars in 1483 bytes} [Sat Oct 15 09:25:47 2022] PUT /identity/v3/projects/3e406b00fdd045668989289a7bf991d3/users/19d5eb7b7dda46d39b055bc465893ccc/roles/3248f0268738489b883041851d681bb7 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:25:47.678449 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7384dc8-17d2-4d24-b369-48b9fd349d71 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:47.679004 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7384dc8-17d2-4d24-b369-48b9fd349d71 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:47.679286 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7384dc8-17d2-4d24-b369-48b9fd349d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:47.707261 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-c7384dc8-17d2-4d24-b369-48b9fd349d71 None None] MFA Rules not processed for user `19d5eb7b7dda46d39b055bc465893ccc`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:25:47.708505 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-c7384dc8-17d2-4d24-b369-48b9fd349d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:25:47.770643 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1335/2655] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:25:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:25:47.782291 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddb0be7-edcf-46ad-b9a5-dff9634aa78f admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:47.784043 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-2ddb0be7-edcf-46ad-b9a5-dff9634aa78f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:25:47.805949 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:25:47.843240 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddb0be7-edcf-46ad-b9a5-dff9634aa78f admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:47.846881 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02647fc-f0ea-4124-976c-32b525841cea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:47.847477 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02647fc-f0ea-4124-976c-32b525841cea service cinder] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:47.847758 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02647fc-f0ea-4124-976c-32b525841cea service cinder] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:47.848034 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02647fc-f0ea-4124-976c-32b525841cea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:47.856121 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02647fc-f0ea-4124-976c-32b525841cea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:47.857570 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02647fc-f0ea-4124-976c-32b525841cea service cinder] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:47.866224 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1321/2656] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:25:47 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:25:50.412199 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7384dc8-17d2-4d24-b369-48b9fd349d71 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:50.430933 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:25:50.475651 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7384dc8-17d2-4d24-b369-48b9fd349d71 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:50.480143 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21db71c-ea36-4391-9c05-880b64df0af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:50.480770 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21db71c-ea36-4391-9c05-880b64df0af8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:50.481051 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21db71c-ea36-4391-9c05-880b64df0af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:50.481329 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21db71c-ea36-4391-9c05-880b64df0af8 admin admin] PATH_INFO: `/v3/users/19d5eb7b7dda46d39b055bc465893ccc` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:50.482464 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21db71c-ea36-4391-9c05-880b64df0af8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19d5eb7b7dda46d39b055bc465893ccc)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:50.484371 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21db71c-ea36-4391-9c05-880b64df0af8 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:50.528650 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b21db71c-ea36-4391-9c05-880b64df0af8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19d5eb7b7dda46d39b055bc465893ccc'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:50.529292 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b21db71c-ea36-4391-9c05-880b64df0af8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19d5eb7b7dda46d39b055bc465893ccc'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:50.536576 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b21db71c-ea36-4391-9c05-880b64df0af8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19d5eb7b7dda46d39b055bc465893ccc'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:50.548176 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b21db71c-ea36-4391-9c05-880b64df0af8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19d5eb7b7dda46d39b055bc465893ccc'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:50.555315 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b21db71c-ea36-4391-9c05-880b64df0af8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19d5eb7b7dda46d39b055bc465893ccc'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:50.561858 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1336/2657] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 09:25:50 2022] DELETE /identity/v3/users/19d5eb7b7dda46d39b055bc465893ccc => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:25:50.566434 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde6d52e-ca94-4208-b19e-800770d0637a None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:50.566913 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde6d52e-ca94-4208-b19e-800770d0637a None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:50.567011 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde6d52e-ca94-4208-b19e-800770d0637a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:50.586525 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-fde6d52e-ca94-4208-b19e-800770d0637a None None] MFA Rules not processed for user `6a470f28aa544b3cb42de175f4e32d94`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:25:50.587718 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-fde6d52e-ca94-4208-b19e-800770d0637a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:25:50.645023 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1322/2658] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:25:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:25:50.654382 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21db71c-ea36-4391-9c05-880b64df0af8 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:50.655834 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-b21db71c-ea36-4391-9c05-880b64df0af8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:25:50.713151 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21db71c-ea36-4391-9c05-880b64df0af8 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:50.716711 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf9e68f-85d3-4b62-83a8-7090ef00b12a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:50.717341 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf9e68f-85d3-4b62-83a8-7090ef00b12a service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:50.717651 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf9e68f-85d3-4b62-83a8-7090ef00b12a service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:50.717952 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf9e68f-85d3-4b62-83a8-7090ef00b12a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:50.726797 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf9e68f-85d3-4b62-83a8-7090ef00b12a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:50.728218 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf9e68f-85d3-4b62-83a8-7090ef00b12a service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:50.737451 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1337/2659] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:25:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:25:50.992945 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde6d52e-ca94-4208-b19e-800770d0637a None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:50.994418 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-fde6d52e-ca94-4208-b19e-800770d0637a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:25:51.007108 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde6d52e-ca94-4208-b19e-800770d0637a None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:51.011593 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79914fde-ba84-49e6-a4e8-53dca2a69a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:51.012221 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79914fde-ba84-49e6-a4e8-53dca2a69a16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:51.012497 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79914fde-ba84-49e6-a4e8-53dca2a69a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:51.012763 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79914fde-ba84-49e6-a4e8-53dca2a69a16 admin admin] PATH_INFO: `/v3/projects/3e406b00fdd045668989289a7bf991d3` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:51.013895 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79914fde-ba84-49e6-a4e8-53dca2a69a16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e406b00fdd045668989289a7bf991d3)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:51.015896 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79914fde-ba84-49e6-a4e8-53dca2a69a16 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:51.063142 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-79914fde-ba84-49e6-a4e8-53dca2a69a16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e406b00fdd045668989289a7bf991d3', 'request_id': 'req-79914fde-ba84-49e6-a4e8-53dca2a69a16'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:51.070502 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-79914fde-ba84-49e6-a4e8-53dca2a69a16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e406b00fdd045668989289a7bf991d3', 'request_id': 'req-79914fde-ba84-49e6-a4e8-53dca2a69a16'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:51.079621 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-79914fde-ba84-49e6-a4e8-53dca2a69a16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e406b00fdd045668989289a7bf991d3', 'request_id': 'req-79914fde-ba84-49e6-a4e8-53dca2a69a16'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:51.090101 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-79914fde-ba84-49e6-a4e8-53dca2a69a16 admin admin] The token cache is being invalidate because project 3e406b00fdd045668989289a7bf991d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 09:25:51.090390 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-79914fde-ba84-49e6-a4e8-53dca2a69a16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:51.091010 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-79914fde-ba84-49e6-a4e8-53dca2a69a16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:25:51.092526 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1323/2660] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 09:25:50 2022] DELETE /identity/v3/projects/3e406b00fdd045668989289a7bf991d3 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:25:53.734503 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf9e68f-85d3-4b62-83a8-7090ef00b12a service neutron] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:53.743688 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf9e68f-85d3-4b62-83a8-7090ef00b12a service neutron] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:53.781934 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:25:53.794936 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bca53f1-0f96-4ec1-b4a9-7e3e38ca4a13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:53.795535 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca53f1-0f96-4ec1-b4a9-7e3e38ca4a13 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:53.795817 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca53f1-0f96-4ec1-b4a9-7e3e38ca4a13 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:53.796093 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca53f1-0f96-4ec1-b4a9-7e3e38ca4a13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:53.797843 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-7bca53f1-0f96-4ec1-b4a9-7e3e38ca4a13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:25:54.005748 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bca53f1-0f96-4ec1-b4a9-7e3e38ca4a13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:54.007187 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bca53f1-0f96-4ec1-b4a9-7e3e38ca4a13 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:54.027412 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1338/2661] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:25:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:25:55.215143 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79914fde-ba84-49e6-a4e8-53dca2a69a16 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:55.216538 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-79914fde-ba84-49e6-a4e8-53dca2a69a16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:25:55.229275 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79914fde-ba84-49e6-a4e8-53dca2a69a16 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:55.232777 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1ccefd-85a8-429e-ae5f-8876bbd29708 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:55.233378 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1ccefd-85a8-429e-ae5f-8876bbd29708 service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:55.233662 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1ccefd-85a8-429e-ae5f-8876bbd29708 service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:55.233968 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1ccefd-85a8-429e-ae5f-8876bbd29708 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:55.236279 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-1c1ccefd-85a8-429e-ae5f-8876bbd29708 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:25:55.249242 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1ccefd-85a8-429e-ae5f-8876bbd29708 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:55.250694 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1ccefd-85a8-429e-ae5f-8876bbd29708 service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:55.261446 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1324/2662] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:25:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:25:57.560023 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc99e3-b5a4-48d0-80bb-93e3ad075995 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:57.560515 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc99e3-b5a4-48d0-80bb-93e3ad075995 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:57.560617 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc99e3-b5a4-48d0-80bb-93e3ad075995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:57.579044 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-60dc99e3-b5a4-48d0-80bb-93e3ad075995 None None] MFA Rules not processed for user `04b8dc0532de441db59bd4fe465a8c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:25:57.580257 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-60dc99e3-b5a4-48d0-80bb-93e3ad075995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:25:57.633136 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1339/2663] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:25:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:25:57.936450 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1ccefd-85a8-429e-ae5f-8876bbd29708 service neutron] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:25:57.945032 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1ccefd-85a8-429e-ae5f-8876bbd29708 service neutron] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:25:57.948600 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aa17b3-c6ae-47e9-bbf8-d0a560ba5de6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:25:57.949190 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa17b3-c6ae-47e9-bbf8-d0a560ba5de6 service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:25:57.949474 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa17b3-c6ae-47e9-bbf8-d0a560ba5de6 service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:25:57.949781 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa17b3-c6ae-47e9-bbf8-d0a560ba5de6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:25:57.951482 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-49aa17b3-c6ae-47e9-bbf8-d0a560ba5de6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:25:57.961346 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aa17b3-c6ae-47e9-bbf8-d0a560ba5de6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:25:57.962815 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aa17b3-c6ae-47e9-bbf8-d0a560ba5de6 service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:25:57.971180 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1325/2664] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:25:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:26:07.626680 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=73 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:26:07.967131 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:26:14.387051 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dc99e3-b5a4-48d0-80bb-93e3ad075995 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:26:14.388775 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-60dc99e3-b5a4-48d0-80bb-93e3ad075995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:26:14.406181 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dc99e3-b5a4-48d0-80bb-93e3ad075995 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:26:14.412351 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b6c017-a2bc-48ff-a24a-b87f15be2a5d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0a8af5cadc4cc1ab73e8eb2a20bb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:26:14.413019 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6c017-a2bc-48ff-a24a-b87f15be2a5d service nova] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:26:14.413299 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6c017-a2bc-48ff-a24a-b87f15be2a5d service nova] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:26:14.413573 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6c017-a2bc-48ff-a24a-b87f15be2a5d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:26:14.415326 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-e4b6c017-a2bc-48ff-a24a-b87f15be2a5d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:26:14.427746 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b6c017-a2bc-48ff-a24a-b87f15be2a5d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:26:14.429563 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b6c017-a2bc-48ff-a24a-b87f15be2a5d service nova] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:26:14.440384 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1340/2665] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:26:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 09:26:24.436543 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=6 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:27:54.038413 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa50da20-15f3-442a-b4f3-bf85fdf849e5 None None] REQUEST_METHOD: `POST` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:27:54.038955 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa50da20-15f3-442a-b4f3-bf85fdf849e5 None None] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:27:54.039584 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa50da20-15f3-442a-b4f3-bf85fdf849e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:27:54.060395 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.auth.core [None req-fa50da20-15f3-442a-b4f3-bf85fdf849e5 None None] MFA Rules not processed for user `04b8dc0532de441db59bd4fe465a8c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=88060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 09:27:54.061611 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-fa50da20-15f3-442a-b4f3-bf85fdf849e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:27:54.068512 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1326/2666] 10.222.0.30 () {58 vars in 958 bytes} [Sat Oct 15 09:27:54 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 09:28:04.055150 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=3 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:28:54.169011 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b6c017-a2bc-48ff-a24a-b87f15be2a5d service nova] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:28:54.170419 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-e4b6c017-a2bc-48ff-a24a-b87f15be2a5d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:28:54.235674 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b6c017-a2bc-48ff-a24a-b87f15be2a5d service nova] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:28:54.239211 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d73213c-4883-42a4-be14-09a4c8624928 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a649c5fedad847fb9e784e779da3a24e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:28:54.239808 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d73213c-4883-42a4-be14-09a4c8624928 service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:28:54.240087 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d73213c-4883-42a4-be14-09a4c8624928 service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:28:54.240360 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d73213c-4883-42a4-be14-09a4c8624928 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:28:54.242103 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-1d73213c-4883-42a4-be14-09a4c8624928 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:28:54.252594 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d73213c-4883-42a4-be14-09a4c8624928 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:28:54.254041 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d73213c-4883-42a4-be14-09a4c8624928 service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:28:54.263303 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1341/2667] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:28:54 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:28:58.462388 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa50da20-15f3-442a-b4f3-bf85fdf849e5 None None] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:28:58.464440 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-fa50da20-15f3-442a-b4f3-bf85fdf849e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:28:58.476909 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa50da20-15f3-442a-b4f3-bf85fdf849e5 None None] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:28:58.481061 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce63c1e2-9600-4a59-83fd-604babe76321 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:28:58.481649 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce63c1e2-9600-4a59-83fd-604babe76321 service neutron] REQUEST_METHOD: `GET` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:28:58.481951 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce63c1e2-9600-4a59-83fd-604babe76321 service neutron] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:28:58.482218 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce63c1e2-9600-4a59-83fd-604babe76321 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:28:58.483843 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-ce63c1e2-9600-4a59-83fd-604babe76321 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:28:58.492922 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce63c1e2-9600-4a59-83fd-604babe76321 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:28:58.494381 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce63c1e2-9600-4a59-83fd-604babe76321 service neutron] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:28:58.502622 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1327/2668] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:28:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:29:02.858519 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d73213c-4883-42a4-be14-09a4c8624928 service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:29:02.868040 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d73213c-4883-42a4-be14-09a4c8624928 service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:29:02.871489 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4cc020-9207-40da-a3e0-6b07b3483719 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b11e349e4cb4558a4c75d843effc0cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:29:02.872071 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4cc020-9207-40da-a3e0-6b07b3483719 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:29:02.872347 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4cc020-9207-40da-a3e0-6b07b3483719 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:29:02.872614 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4cc020-9207-40da-a3e0-6b07b3483719 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:29:02.874835 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-cc4cc020-9207-40da-a3e0-6b07b3483719 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:29:02.956741 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4cc020-9207-40da-a3e0-6b07b3483719 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:29:02.958201 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4cc020-9207-40da-a3e0-6b07b3483719 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:29:02.976657 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1342/2669] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:29:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:29:08.499149 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=6 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:29:09.174299 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce63c1e2-9600-4a59-83fd-604babe76321 service neutron] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:29:09.175824 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-ce63c1e2-9600-4a59-83fd-604babe76321 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:29:09.190863 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce63c1e2-9600-4a59-83fd-604babe76321 service neutron] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:29:09.195722 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7eaee2-a32f-4e34-a928-70de6d9ac359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:29:09.196361 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7eaee2-a32f-4e34-a928-70de6d9ac359 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:29:09.196625 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7eaee2-a32f-4e34-a928-70de6d9ac359 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:29:09.196900 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7eaee2-a32f-4e34-a928-70de6d9ac359 admin admin] PATH_INFO: `/v3/users/04b8dc0532de441db59bd4fe465a8c0a` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:29:09.198110 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7eaee2-a32f-4e34-a928-70de6d9ac359 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04b8dc0532de441db59bd4fe465a8c0a)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:29:09.200130 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7eaee2-a32f-4e34-a928-70de6d9ac359 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:29:09.245821 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-cc7eaee2-a32f-4e34-a928-70de6d9ac359 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04b8dc0532de441db59bd4fe465a8c0a'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:29:09.246491 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-cc7eaee2-a32f-4e34-a928-70de6d9ac359 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04b8dc0532de441db59bd4fe465a8c0a'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:29:09.258697 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-cc7eaee2-a32f-4e34-a928-70de6d9ac359 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04b8dc0532de441db59bd4fe465a8c0a'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:29:09.266720 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-cc7eaee2-a32f-4e34-a928-70de6d9ac359 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04b8dc0532de441db59bd4fe465a8c0a'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:29:09.267366 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-cc7eaee2-a32f-4e34-a928-70de6d9ac359 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04b8dc0532de441db59bd4fe465a8c0a'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:29:09.279773 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1328/2670] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 09:29:09 2022] DELETE /identity/v3/users/04b8dc0532de441db59bd4fe465a8c0a => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:29:09.450708 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4cc020-9207-40da-a3e0-6b07b3483719 service neutron] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:29:09.462154 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4cc020-9207-40da-a3e0-6b07b3483719 service neutron] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:29:09.517770 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1624a834-d5aa-40c7-9f03-453b407f228d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:29:09.518331 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1624a834-d5aa-40c7-9f03-453b407f228d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:29:09.518607 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1624a834-d5aa-40c7-9f03-453b407f228d admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:29:09.518879 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1624a834-d5aa-40c7-9f03-453b407f228d admin admin] PATH_INFO: `/v3/projects/c6262387226346b7a736e34171afb4de` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:29:09.519960 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1624a834-d5aa-40c7-9f03-453b407f228d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6262387226346b7a736e34171afb4de)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:29:09.521866 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1624a834-d5aa-40c7-9f03-453b407f228d admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:29:09.523229 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:29:09.566959 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-1624a834-d5aa-40c7-9f03-453b407f228d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6262387226346b7a736e34171afb4de', 'request_id': 'req-1624a834-d5aa-40c7-9f03-453b407f228d'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:29:09.572985 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-1624a834-d5aa-40c7-9f03-453b407f228d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6262387226346b7a736e34171afb4de', 'request_id': 'req-1624a834-d5aa-40c7-9f03-453b407f228d'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:29:09.580940 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-1624a834-d5aa-40c7-9f03-453b407f228d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6262387226346b7a736e34171afb4de', 'request_id': 'req-1624a834-d5aa-40c7-9f03-453b407f228d'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:29:09.591214 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-1624a834-d5aa-40c7-9f03-453b407f228d admin admin] The token cache is being invalidate because project c6262387226346b7a736e34171afb4de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 09:29:09.591538 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-1624a834-d5aa-40c7-9f03-453b407f228d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:29:09.592152 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-1624a834-d5aa-40c7-9f03-453b407f228d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:29:09.593609 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1343/2671] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 09:29:09 2022] DELETE /identity/v3/projects/c6262387226346b7a736e34171afb4de => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 09:29:09.597557 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7eaee2-a32f-4e34-a928-70de6d9ac359 admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:29:09.598981 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-cc7eaee2-a32f-4e34-a928-70de6d9ac359 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:29:09.612353 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7eaee2-a32f-4e34-a928-70de6d9ac359 admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:29:09.668994 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75154ede-8320-4221-90a4-1b7e6abafb6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:29:09.669660 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75154ede-8320-4221-90a4-1b7e6abafb6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:29:09.669976 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75154ede-8320-4221-90a4-1b7e6abafb6a admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:29:09.670251 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75154ede-8320-4221-90a4-1b7e6abafb6a admin admin] PATH_INFO: `/v3/users/cdfdfa5581444cb7a27e693e5b622da8` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:29:09.671377 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75154ede-8320-4221-90a4-1b7e6abafb6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdfdfa5581444cb7a27e693e5b622da8)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:29:09.673298 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75154ede-8320-4221-90a4-1b7e6abafb6a admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:29:09.718431 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-75154ede-8320-4221-90a4-1b7e6abafb6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdfdfa5581444cb7a27e693e5b622da8'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:29:09.719072 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-75154ede-8320-4221-90a4-1b7e6abafb6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdfdfa5581444cb7a27e693e5b622da8'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:29:09.730759 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-75154ede-8320-4221-90a4-1b7e6abafb6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdfdfa5581444cb7a27e693e5b622da8'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:29:09.738347 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-75154ede-8320-4221-90a4-1b7e6abafb6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdfdfa5581444cb7a27e693e5b622da8'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:29:09.738960 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-75154ede-8320-4221-90a4-1b7e6abafb6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdfdfa5581444cb7a27e693e5b622da8'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:29:09.741089 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:29:09.753920 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1329/2672] 10.222.0.30 () {58 vars in 1224 bytes} [Sat Oct 15 09:29:09 2022] DELETE /identity/v3/users/cdfdfa5581444cb7a27e693e5b622da8 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 09:29:09.889960 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1624a834-d5aa-40c7-9f03-453b407f228d admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:29:09.891470 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-1624a834-d5aa-40c7-9f03-453b407f228d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:29:09.921981 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75154ede-8320-4221-90a4-1b7e6abafb6a admin admin] Authenticating user token {{(pid=88060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 09:29:09.923546 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.fernet_utils [None req-75154ede-8320-4221-90a4-1b7e6abafb6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:29:09.976187 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1624a834-d5aa-40c7-9f03-453b407f228d admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:29:10.006398 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75154ede-8320-4221-90a4-1b7e6abafb6a admin admin] Validating token access rules against request {{(pid=88060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 09:29:10.010980 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54118d6-1bf6-4049-b3fd-77fc892f1b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a470f28aa544b3cb42de175f4e32d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd200ced7ea7746bab37279a9888da771', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:29:10.011614 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54118d6-1bf6-4049-b3fd-77fc892f1b79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:29:10.011897 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54118d6-1bf6-4049-b3fd-77fc892f1b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:29:10.012173 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54118d6-1bf6-4049-b3fd-77fc892f1b79 admin admin] PATH_INFO: `/v3/projects/0f0556a66f62422d877e74b0b30bd1c2` {{(pid=88060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:29:10.013355 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54118d6-1bf6-4049-b3fd-77fc892f1b79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f0556a66f62422d877e74b0b30bd1c2)` {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:29:10.015318 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54118d6-1bf6-4049-b3fd-77fc892f1b79 admin admin] RBAC: Authorization granted {{(pid=88060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:29:10.036174 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e8d42e-bd8f-4711-9835-10f6aea06813 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38986ca2612c4fc59d960cd693834f9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dfceb59b3794439a4586b3cbe408f0e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 09:29:10.036780 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e8d42e-bd8f-4711-9835-10f6aea06813 service placement] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 09:29:10.037063 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e8d42e-bd8f-4711-9835-10f6aea06813 service placement] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 09:29:10.037337 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e8d42e-bd8f-4711-9835-10f6aea06813 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 09:29:10.039172 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-f3e8d42e-bd8f-4711-9835-10f6aea06813 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 09:29:10.053214 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e8d42e-bd8f-4711-9835-10f6aea06813 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 09:29:10.054830 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e8d42e-bd8f-4711-9835-10f6aea06813 service placement] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 09:29:10.064537 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-d54118d6-1bf6-4049-b3fd-77fc892f1b79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f0556a66f62422d877e74b0b30bd1c2', 'request_id': 'req-d54118d6-1bf6-4049-b3fd-77fc892f1b79'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:29:10.065895 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1344/2673] 10.222.0.30 () {60 vars in 1365 bytes} [Sat Oct 15 09:29:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 09:29:10.072772 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-d54118d6-1bf6-4049-b3fd-77fc892f1b79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f0556a66f62422d877e74b0b30bd1c2', 'request_id': 'req-d54118d6-1bf6-4049-b3fd-77fc892f1b79'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:29:10.080616 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-d54118d6-1bf6-4049-b3fd-77fc892f1b79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f0556a66f62422d877e74b0b30bd1c2', 'request_id': 'req-d54118d6-1bf6-4049-b3fd-77fc892f1b79'} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:29:10.091964 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-d54118d6-1bf6-4049-b3fd-77fc892f1b79 admin admin] The token cache is being invalidate because project 0f0556a66f62422d877e74b0b30bd1c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 09:29:10.092245 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-d54118d6-1bf6-4049-b3fd-77fc892f1b79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:29:10.092888 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG keystone.notifications [None req-d54118d6-1bf6-4049-b3fd-77fc892f1b79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 09:29:10.094568 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: [pid: 88060|app: 0|req: 1330/2674] 10.222.0.30 () {58 vars in 1233 bytes} [Sat Oct 15 09:29:09 2022] DELETE /identity/v3/projects/0f0556a66f62422d877e74b0b30bd1c2 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 09:29:20.061215 dv-cinder-856971-3-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=82,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 09:29:20.088154 dv-cinder-856971-3-iscsi devstack@keystone.service[88060]: DEBUG dbcounter [-] [88060] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}